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Abstract 


Human Factors in Web Authentication 
by 

Chris K. Karlof 

Doctor of Philosophy in Computer Science 
University of California, Berkeley 
Professor Doug Tygar, Co-Chair 
Professor David Wagner, Co-Chair 

This dissertation endeavors to improve the security of user authentication on the World 
Wide Web. One threat to Web authentication is phishing, a social engineering attack that 
solicits users’ authentication credentials by spoofing the login page of a trusted Web site. 
We identify human psychological tendencies that make users susceptible to phishing at¬ 
tacks and apply these insights to develop design principles for conditioned-safe ceremonies. 
Conditioned-safe ceremonies are security protocols that deliberately condition users to re- 
flexively act in ways that protect them from attacks. Our formulation of conditioned-safe 
ceremonies draws on several ideas and lessons learned from the human factors and human 
reliability community: forcing functions, defense in depth, and the use of human tenden¬ 
cies, such as rule-based decision making. 

We apply these principles to develop a conditioned-safe ceremony based on email for 
initializing credentials in machine authentication schemes. We evaluated our email cere¬ 
mony with a user study of 200 participants. We simulated attacks against the users and 
found that our email ceremony was significantly more secure than a comparable one based 
on challenge questions. We found evidence that conditioning helped the email users resist 
attacks, but contributed towards making challenge question users more vulnerable. 

We also address stronger social engineering threats against Web authentication, e.g., 
pharming. We describe a new attack against Web authentication we call dynamic pharm- 
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ing. Dynamic pharming works by hijacking DNS and sending the victim’s browser ma¬ 
licious Javascript, which then exploits DNS rebinding vulnerabilities and the name-based 
same-origin policy to hijack a legitimate session after authentication has taken place. To 
resist dynamic pharming attacks, we propose two locked same-origin policies for Web 
browsers. In contrast to the legacy same-origin policy, which enforces access control in 
browsers using domain names, the locked same-origin policies enforce access control us¬ 
ing servers’ X.509 certificates and public keys. We evaluate the security and deployability 
of our approaches and show how browsers can deploy these policies today to substantially 
increase their resistance to pharming attacks and provide a foundation for the development 
of pharming resistant authentication mechanisms. 


Professor Doug Tygar 
Dissertation Committee Co-Chair 


Professor David Wagner 
Dissertation Committee Co-Chair 
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Chapter 1 
Introduction 


This dissertation argues that since computer security mechanisms can have strong ef¬ 
fects on users’ behavior and choices, we must judiciously design these mechanisms such 
that human psychological tendencies help users resist attacks, rather than make them more 
vulnerable, as many current mechanisms do. Our focus is on user authentication schemes 
on World Wide Web. Despite the development of numerous cryptographic authentication 
mechanisms, user authentication on the World Wide Web remains firmly saddled in the 
1970s: the vast majority of users authenticate themselves to Web sites by sending a plain¬ 
text password over the network. Although widespread deployment of the Secure Sockets 
Layer (SSL) helps protect password authentication against passive eavesdropping attacks, 
it does little to help users resist more devious threats, such as phishing [6], A phishing 
attack is a type of social engineering attack, where an adversary lures an unsuspecting In¬ 
ternet user to a Web site posing as a trustworthy business with the goal of stealing sensitive 
personal information, e.g., the user’s password. Phishers commonly lure victims by send¬ 
ing an email containing a warning about a “problem” which requires immediate attention, 
along with a link the user can click to take action. If a user clicks on the link, she will 
reach the phishing site. A phishing attack typically prompts the user to enter some per¬ 
sonal information, such as a login name, password, or social security number, before the 
“problem” with her account can be addressed. Phishing attacks have become widespread 
over the last decade, and their success has created a multi-million dollar underground econ¬ 
omy [33, 117]. Our goal is to develop authentication mechanisms resistant to phishing and 
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other social engineering attacks that target users' Web accounts. 

One explanation for the success of phishing attacks is a human psychological tendency 
to develop automatic responses to situations we encounter more than once. Our brains 
tend to classify stimuli according to a few key features, and if one or more features match 
stimuli we have encountered in the past, we often respond mindlessly with the action that 
we learned was most appropriate. Psychologist Robert Cialdini calls these click-whirr 
responses [16]. Cialdini compares these automatic responses to pre-recorded tapes in our 
head, and uses “click-whirr” to evoke the sound a tape machine makes after pressing “play”. 
As the world becomes more intricate and variable, we increasingly rely on click-whirr 
responses. Without click-whirr responses, we would spend most of our time appraising 
and analyzing mundane situations in our daily lives. Philosopher Alfred North Whitehead 
recognized this when he asserted “civilization advances by the extending the number of 
operations we can perform without thinking about them [131].” 

As we become more dependent on click-whirr responses to navigate our daily lives, 
some have learned to exploit this behavior. Stored schemas of past recollections and sen¬ 
sory inputs tend to only contain evidence of how a previously encountered situation should 
appear - they rarely store information about how that situation should not appear [100]. 
Salesman, fund raisers, and con men can create situations containing the stimuli necessary 
to trigger the desired click-whirr response, even though less visible features may differ 
substantially from past situations. For example, people tend to obey a person in a uniform, 
regardless of whether that person has any real authority. 

The designers of many current Web authentication mechanisms, such as passwords, 
have all but ignored this fundamental psychological phenomenon. The most common Web 
authentication technique in use today is password authentication via an HTML form, where 
a user types her password directly into a Web page from the site to which she wishes to au¬ 
thenticate herself. Social engineering attacks on the Internet, such as phishing, have largely 
been successful because the Web is fertile ground for mimicry, and password authentication 
can condition users to fall for these attacks. Since a wide range of Web sites require a user 
to log in before she can do something interesting, many users have developed a click-whirr 
response to login forms and will automatically enter their login credentials on any page 
which on the surface appears familiar, legitimate, or trustworthy. 
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This dissertation makes two main contributions to the development of Web authen¬ 
tication mechanisms that resist phishing and other social engineering attacks. First, we 
introduce the notion of a conditioned-safe ceremony , a new approach for developing user¬ 
centric authentication and security mechanisms. Second, we address two security issues 
with machine authentication, a procedure that authenticates a user’s computer, as opposed 
to herself. We discuss these contributions further in the following sections. 

1.1 Conditioned-safe ceremonies 

Web authentication is an example of a ceremony. A ceremony is similar to the con¬ 
ventional notion of a network protocol, except that a ceremony explicitly includes human 
participants as nodes in the network, distinct from the computers and devices they use [29]. 1 
Communications between human nodes and other nodes in the ceremony are usually not 
via network connections, but instead through user interfaces, face-to-face interactions, or 
peripheral devices. 

Although current Web authentication ceremonies, such as password authentication, are 
vulnerable to attacks that exploit human psychological tendencies such as click-whirr re¬ 
sponses, we argue that these tendencies are not something that we necessarily must avoid 
or suppress in ceremony design. Instead, in Chapter 3, we hypothesize that we can build 
ceremonies that turn the tables on adversaries and take advantage of these psychologi¬ 
cal tendencies to help users resist social engineering attacks. We identify several ways 
in which many current security mechanisms and ceremonies have disregarded human ten¬ 
dencies, and present a model for user behavior during social engineering attacks based 
on psychological research on human performance and error, such as Jens Rasmussen’s 
skill-rule-knowledge framework [99] and James Reason’s Generic Error-Modeling Sys¬ 
tem [100]. Based on this model, we introduce the notion of a conditioned-safe ceremony. 
A conditioned-safe ceremony is one that deliberately conditions users to reflexively act in 
ways that protect them from attacks. Many existing ceremonies require users to make dif¬ 
ficult and subtle security decisions or respond to exceptional situations to resist attacks. In 
1 The term ceremony was first coined for this purpose by Jesse Walker [29]. 
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contrast, if a user of a conditioned-safe ceremony simply performs the same actions during 
an attack as she usually performs under normal conditions, she will resist - even if she 
is unaware of the threat. Our formulation of conditioned-safe ceremonies in Section 3.3 
draws on several ideas and lessons learned from the human factors and human reliability 
community: forcing functions, defense in depth, and the use of human tendencies, such as 
rule-based decision making. 

1.2 Securing machine authentication 

To address the precipitous rise in social engineering attacks on the Internet, the Fed¬ 
eral Financial Institutions Examination Council declared in October 2005 that passwords 
alone are “inadequate for high-risk transactions” [20]. In response, many institutions use 
machine authentication , which authenticates a user’s computer , in addition to password au¬ 
thentication, which authenticates the user herself. For example, one widely used approach 
for machine authentication is to set a persistent cookie; since the user’s browser will send 
that cookie every time the user returns to the Web site from that computer, the Web site can 
recognize the user’s computer. To successfully log in, the user must provide her password 
and the user’s browser must present a valid cookie. The intention is to take the human 
“out of the loop” and reduce the system’s dependency on humans’ abilities to detect at¬ 
tacks. Web sites currently using machine authentication include Bank of America [9], ING 
Direct [52] and Vanguard [123]. 

One potential advantage of machine authentication is that it increases the difficulty of 
social engineering attacks against users’ accounts. The problem with password authenti¬ 
cation is that it is too easy for users to reveal their passwords to phishers and other adver¬ 
saries. In one standard classification of authentication schemes, passwords are “something 
you know”; but the problem with using “something you know” for authentication is that 
anything the user knows, she can—and in a nontrivial fraction of cases, will—reveal to an 
adversary. Instead, machine authentication relies on “something your computer knows.” 
This is roughly equivalent to “something you have,” except implemented in software, with¬ 
out requiring the physical hardware tokens normally used to fill that role. 
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To compromise a user’s account, an attacker not only needs to phish the user’s pass¬ 
word, but must also steal the user’s authentication cookie. Although simply requesting a 
user’s password has proven to be a relatively successful attack, pilfering a user’s cookie 
from her machine is not as straightforward; computers are not fooled by social engineering 
attacks. Browsers automatically determine the appropriate cookies for Web requests and 
require little to no user involvement to make security decisions, reducing the chance of 
human error. 

We address two security issues with machine authentication: 1) initializing machine 
authentication credentials on users’ computers, i.e., the registration problem , and 2) se¬ 
curing machine authentication against stronger threat models, such as pharming and active 
attackers. 

1.2.1 The registration problem 

Since users may use more than one computer, machine authentication systems must 
have a registration ceremony to authorize and set authentication cookies on multiple ma¬ 
chines. Unfortunately, this additional functionality potentially brings the human back “in 
the loop” and exposes machine authentication systems to an alternative attack vector. In¬ 
stead of trying to steal authentication cookies directly from a user’s machine, an attacker 
can try to subvert the registration ceremony in a way that grants the attacker a valid cookie 
for the user’s account. Consequently, registration ceremonies must resist these kinds of 
bootstrapping attacks. 

Many machine authentication systems currently deployed by financial Web sites use 
challenge question based registration [9, 52, 123]. A challenge question is a user-specific 
question to which an adversary is unlikely to be able to guess an answer, e.g., “What is 
the name of your favorite teacher?” [30, 65]. Registration based on challenge questions is 
vulnerable to man-in-the-middle (MITM) attacks [113, 144], Since these attacks exploit 
similar human tendencies as attacks against passwords, the security benefits of challenge 
questions over passwords alone may be minimal. 




6 


An email-based registration ceremony. We apply our design principles for conditioned- 
safe ceremonies to develop a registration ceremony for machine authentication based on 
email (Section 4.2). When a user attempts to log in from an unregistered computer, the 
Web site sends her an email containing a single-use HTTPS URL. When the user clicks on 
the registration link, the Web site sets a persistent authentication cookie on the user’s com¬ 
puter. For subsequent logins from that computer, she only needs to complete any supple¬ 
mentary login procedures, e.g., enter her username and password. We discuss email-based 
registration further in Section 4.2. 

A user study of registration ceremonies. To evaluate our email based registration cer¬ 
emony, we conducted a user study with 200 participants to compare the security of email 
registration to the security of registration based on challenge questions (Chapter 5). We 
designed our study to be as ecologically valid as possible: we employed deception, did 
not use a laboratory environment, and attempted to create an experience of risk. We sim¬ 
ulated social engineering attacks against the users and found email based registration was 
significantly more secure against our attacks (Table 5.4). Our simulated attacks succeeded 
against 93% of challenge question users, but succeeded against only 41% of email users. 
We also found evidence that conditioning helped email registration users resist our sim¬ 
ulated attacks, but contributed towards making challenge question users more vulnerable. 
We asked users to complete an exit survey after they finished the study, and we analyze the 
results in Sections 5.3 and 5.4. 

1.2.2 Dynamic pharming attacks and the locked same-origin policies 

Although phishing has been one of the most prevalent types of social engineering at¬ 
tacks on the Web, stronger attacks, such as pharming , are a growing threat [4, 66, 97, 115, 
120, 121]. In a pharming attack [91], the adversary subverts the domain-name lookup sys¬ 
tem (DNS), which is used to resolve domain names to IP addresses. In this attack, the 
DNS infrastructure is compromised so that DNS queries for the victim site’s domain (say, 
google . com) return an attacker-controlled IP address. Pharming attacks are particularly 
devious because the browser’s URL bar will display the domain name of the site the user 
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intended to visit. 

Dynamic pharming attacks. We describe a new type of DNS attack against Web au¬ 
thentication we call dynamic pharming. In a dynamic pharming attack, the adversary ini¬ 
tially delivers a Web document containing malicious Javascript code to the victim, and 
then forces the victim’s browser to connect to the legitimate server in a separate window or 
frame. The adversary waits for the victim to authenticate herself to the legitimate server, 
and then uses the malicious Javascript to hijack the victim’s authenticated session. 

Dynamic pharming takes advantage of how browsers currently implement the same- 
origin policy. The same-origin policy prohibits a Web object from one site from accessing 
Web objects served from a different site. Browsers currently enforce this by checking that 
the two objects’ originating domain names, ports, and protocols match. However, when 
an adversary controls the domain name mapping, the legacy same-origin policy does not 
provide strong isolation between Web objects co-executing in a user’s browser. In a dy¬ 
namic pharming attack, malicious Javascript from the pharmer and content from the legiti¬ 
mate server both appear to have the same “origin” (i.e., same domain, port, and protocol), 
and the browser allows the Javascript to access to the user’s authenticated session. As a 
result, the attacker can gain complete control of the session, enabling her to eavesdrop 
on sensitive content, forge transactions, sniff secondary passwords, etc. Since dynamic 
pharming hijacks users’ sessions after authentication completes, irrespective of the authen¬ 
tication mechanism, it can be used to compromise even the strongest Web authentication 
schemes currently known, including all forms of machine authentication. We present dy¬ 
namic pharming in more detail in Chapter 6. 

The locked same-origin policies. Since dynamic pharming hijacks a user’s session af¬ 
ter initial authentication completes, it is unlikely any future Web authentication ceremony 
developed for currently deployed browsers will resist dynamic pharming either. To resist 
dynamic pharming, we address the root of the problem: we upgrade the browser’s same- 
origin policy. We propose two locked same-origin policies : instead of comparing domain 
names to enforce access control, our policies enforce access control for Web objects re¬ 
trieved over SSL by using servers’ public keys and X.509 certificates. We refer to Web 
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objects retrieved over SSL as locked Web objects because the browser can clearly asso¬ 
ciate the public key and X.509 certificate of server hosting the object with the object. Our 
first proposal, the weak locked same-origin policy, isolates a domain’s locked Web objects 
with valid certificate chains from objects with invalid chains. This enables browsers to 
distinguish a legitimate server using a valid certificate from pharmers using invalid cer¬ 
tificates, such as self-signed certificates or certificates with CN/domain mismatches. Our 
second proposal, the strong locked same-origin policy, enforces access control using cryp¬ 
tographic identity, namely Web sites’ public SSL keys. In the strong locked same-origin 
policy, the browser compares the public keys it associates with locked Web objects; access 
is granted only if they match. In Chapter 7, show how our policies substantially increase 
browsers’ resistance to pharming attacks and provide a solid foundation for developing 
pharming resistant machine authentication. 

We evaluate our policies in terms of deployability, meaning how well they interoperate 
with existing Web servers. Based on the results of a study of 14651 SSL domains, we found 
strong evidence that the weak locked same-origin policy can replace the legacy same-origin 
policy today with minimal risk of breaking existing Web sites (Section 7.4). Although we 
did not find similar evidence for the strong locked same-origin policy, we propose a simple, 
incrementally deployable and backwards compatible mechanism for Web sites to opt in 
using policy files (Section 7.5). To opt in, we propose that a Web site should post a policy 
file which enables the site to specify how it would like the browser to enforce the strong 
locked same-origin policy. Policy files also support flexible server configurations and key 
updates. In contrast to the weak locked same-origin policy, the strong locked same-origin 
policy has better security properties, is compatible with sites using self-signed or untrusted 
certificates, and supports subdomain object sharing. 
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Chapter 2 
Preliminaries 

2.1 Background 

2.1.1 HTTP cookies 

HTTP cookies are a general mechanism for Web servers to store and retrieve persis¬ 
tent state on Web clients [94], When a client makes an HTTP request to a server, the 
server has the option of including one or more Set-Cookie headers in its response. The 
Set-Cookie HTTP header allows several optional attributes. The expires attribute 
indicates when a browser should delete the cookie. If the expires attribute is omitted, 
then the cookie is called a session cookie and should be deleted when the user closes the 
Web browser. Cookies with an expires attribute are called persistent cookies. 

The domain attribute specifies the set of HTTP requests for which a browser should 
include the cookie. For example, if the user requests the URL http : / /www. f oo . com/ 
index . html, then a cookie with domain=f oobar . com or domain=www. too . com 
would be included with this request, but a cookie with domain=pics .foo.com would 
not. The default value of the domain attribute is the host name of the server which gen¬ 
erated the cookie response. We refer to a cookie with an explicit domain attribute as a 
domain cookie and a cookie which omits it as a host cookie. 

The secure attribute indicates that the browser should only send the cookie over SSL 
connections. We refer to a cookie including the secure attribute as an SSL-only cookie. 
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Javascript may also access cookies through the document. cookie property. For this 
mode of access, Web browsers use the URL of the document executing the Javascript to 
determine the appropriate cookies. 

Recent browsers support more structured persistent storage mechanisms with function¬ 
ality similar to cookies. Some browser plugin architectures also provide persistent storage 
mechanisms, e.g., Adobe Flash and Google Gears. 

2.1.2 The legacy same-origin policy 

The same-origin policy (SOP) in Web browsers governs access control among different 
Web objects and prohibits a Web object from one origin from reading or modifying Web 
objects from a different origin [87], By Web objects, we mean HTTP cookies, HTML 
documents, images, Javascript, CSS hies, XML files, etc. A common example of “access” 
is Javascript referencing another object. In the remainder of this dissertation, we will use 
“SOP” as an abbreviation for “same-origin policy”. 

Browsers currently consider two objects to have the same origin if the originating 
host, port, and protocol are the same for both Web objects. A Web object can read and 
modify another object only if they have the same origin. For example, Javascript execut¬ 
ing on http : / /www. f oo . com/index . html is allowed to read and modify http : 
/ /www. f oo . com/other . html, but is not allowed to access https : / /www. f oo . 
com/secure . html (different protocol) or http://www.xyz.com/index.html 
(different host). Other examples of “Web object accesses” subject to the SOP include de¬ 
termining which cookies to append to an HTTP request, Javascript document .cookie 
references, and XMLHTTPRequest. 

Note there is a distinction between “access” and “causing to load”. After the browser 
receives an HTML page, it processes dependent requests necessary to render the page, such 
as images, style sheets, etc. These requests can cause the browser to fetch and load a Web 
object from a different domain. However, these requests are not considered violations of 
the SOP. The document can read certain metaproperties of the object (e.g., height, width), 
but the SOP prevents the document from reading or modifying the content of the loaded 
object. 
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2.1.3 Secure Sockets Layer (SSL) and X.509 certificates 

The Secure Sockets Layer (SSL) and its successor. Transport Layer Security (TLS), are 
cryptographic protocols for establishing end-to-end secure channels for Internet traffic [34, 
119]. HTTP over SSL is also known as HTTPS. 

SSL uses X.509 certificates [50] to identify the server participating in the SSL connec¬ 
tion. An X.509 certificate contains the server’s public key, the domain name of the Web site 
(specified in the CN subfield of the certificate), the public key of the issuer of the certificate, 
the time period for which the certificate is valid, and the issuer’s signature over these fields. 
The private key corresponding to a X.509 certificate can be used to sign another certificate, 
and so on, creating a chain of trust. The root of this trust chain is typically a certificate 
authority (CA); Web browsers ship with the certificates of some CAs that are deemed to be 
trusted. 

When the client’s Web browser makes a connection to an SSL enabled Web server over 
HTTPS, the browser must verify the server’s certificate is valid. This involves numerous 
checks, but at a high level the browser must: 

• Verify that every certificate in the chain has a valid signature from its predecessor, 
using the public key of the predecessor, and that the last certificate in the chain is 
from a trusted CA. 

• Verify that the CN field of the first certificate in the chain matches the domain name 
of the Web site the browser intended to visit. 

• Verify every certificate has not expired. 

If any of these checks fail, the browser warns the user and asks the user if it is safe to 
continue. If the user chooses, the user may permit the SSL connection to continue even 
though any or all of these checks have failed. The reason is to ensure compatibility with 
misconhgured certificates and SSL servers. Also, this behavior by browsers allows Web 
sites to use self-signed certificates if they choose, instead of paying a CA for a certificate. 
Unfortunately, asking users whether to continue in such cases is a serious security vulnera¬ 
bility. Researchers have shown that users routinely ignore such security warnings and just 
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click “OK” [11, 24, 135]. In fact, users have become so ambivalent to security warnings, 
one vendor has developed “mouse auto-clicker” software, aptly titled “Press the Freakin 
Button” [95], to automatically click through dialogs like these. Instead of a dialog box, 
IE 7.0 and Firefox 3 use a full page warning within the browser window offering similar 
options (i.e., accept certificate and continue, or cancel connection), although bypassing the 
Firefox 3 warning requires several steps. Unfortunately, studies and anecdotal evidence 
suggest that users will ignore a full page warning as well [12, 108]. Accordingly, we con¬ 
sider a certificate that does not generate any warnings as valid. Otherwise, we consider it 
invalid. 

After the browser validates the server’s certificate, it participates in a cryptographic 
protocol with the server where: 1) the server proves knowledge of the private key corre¬ 
sponding to the public key in the certificate, and 2) they negotiate a session key to encrypt 
and authenticate subsequent traffic between them. Unlike the certificate validation step, if 
there are any errors in this protocol, the browser closes the connection with no chance of 
user override. 

Client-side SSL. The most common usage of SSF is for server authentication, but in the 
SSF specification, a server can also request client-side authentication, where the client also 
presents an X.509 certificate and proves knowledge of the corresponding private key. Using 
client-side SSF, servers can identify a user with her SSF public key and authenticate her 
using the SSF protocol. 

2.2 Threat models 

We consider three broad classes of adversaries, classified according to their capabilities. 

2.2.1 Phishers 

Phishing is a social engineering attack in which an adversary lures an unsuspecting 
Internet user to a Web site posing as a trustworthy business with which the user has a rela¬ 
tionship [6]. The broad goal is identity theft; phishers try to fool Web visitors into revealing 
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their login credentials, sensitive personal information, or credit card numbers with the in¬ 
tent of impersonating their victims for financial gain. Phishers commonly lure victims by 
sending an email containing a warning about a “problem” which requires immediate atten¬ 
tion, along with a link the user can click to take action. If a user clicks on the link, she 
will reach the phishing site. Typically a phishing attack prompts the user to enter some 
personal information, such as a login name, password, or social security number, before 
the “problem” with her account can be addressed. We assume a phisher has the following 
capabilities: 

• Complete control of a Web server with a public IP address. We assume a phisher 
uses a different domain name from the target domain. 

• The ability to send communications such as emails and instant messages to potential 
victims. 

• Mount application-layer man-in-the-middle attacks, representing a legitimate server 
to the victim and proxying input from the victim to the real server as needed. 

There have been relatively few documented cases of application-layer man-in-the-middle 
attacks [7, 127, 128], most likely because of the extra effort required to implement the 
attack. However, researchers have discovered a “Universal Man-in-the-Middle Phishing 
Kit” [75], a hacker toolkit which enables a phisher to easily set up a MITM proxy attack 
against any site she wishes. 

To date, phishers have been the most prevalent class of attacker; however, looking to 
the future, stronger attackers are a growing threat [4, 66, 97, 115, 120, 121], and it seems 
prudent to defend against these more powerful attackers as well, to the extent possible. We 
discuss these stronger threat models next. 

2.2.2 Pharmers 

In a pharming attack [91], the adversary subverts the domain-name lookup system 
(DNS), which is used to resolve domain names to IP addresses. In this attack, the DNS 
infrastructure is compromised so that DNS queries for the victim site’s domain (say, 
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google. com) return an attacker-controlled IP address. We assume a pharmer has all 
the abilities of a phisher, plus: 

• The ability to change DNS records for the target site, such that the victim will resolve 
the target site’s name to the attacker’s IP address. 

We assume the server under the pharmer’s control does not have the same IP address as the 
victim and cannot receive packets destined to the victim’s IP address. 

Pharming can be accomplished via several techniques, including DNS cache poisoning, 
DNS response forgery, modifying a user’s /etc/hosts file, tricking a user to modify her 
DNS settings, or by social engineering attacks against a domain name registry. Pharming 
attacks are particularly devious because the browser’s URL bar will display the domain 
name of the legitimate site, potentially fooling even the most meticulous users. 

Although pharming attacks have been relatively rare in practice, evidence suggests they 
may become a more serious threat in the near future. Recent research has exposed complex 
and subtle dependencies between names and name servers [97] and weaknesses in the DNS 
protocol [66], suggesting the DNS infrastructure is more vulnerable to DNS poisoning 
attacks than previously thought. 

2.2.3 Active attackers 

We assume an active attacker has all the abilities of a pharmer, plus 

• The ability to modify, re-route, drop, and delay all traffic to and from the victim’s IP 
address. 

• The ability to eavesdrop on all traffic to and from the victim's IP address. 

• The ability to mount active, network-layer, man-in-the-middle attacks against the 
victim. 

The ubiquity of public wireless access points and wireless home routers introduces new 
opportunities for launching active attacks. Users are becoming accustomed to accessing 
wireless routers in airports, restaurants, conferences, libraries, and other public spaces. Ad¬ 
versaries can set up malicious wireless routers in these areas that offer free Internet access 



15 


but redirect users to spoofed Web sites [4], Also, many users leave the default password 
and security settings on their wireless home routers unchanged [110]. This enables warkit- 
ting attacks [120, 121], a combination of wardriving and rootkitting, where an adversary 
maliciously alters a router’s configuration over a wireless connection. A related attack is 
where a malicious Web site serves content that scans a visitor’s internal network and com¬ 
promises home routers with default passwords. After the adversary has compromised the 
victim’s router, she can change the router’s settings or overwrite the router’s firmware to 
intercept the victim’s traffic [115]. 

2.2.4 Other assumptions 

For the attacks we present in Chapter 6, we assume that many users will ignore certifi¬ 
cate warnings. Several studies suggest that users routinely ignore and dismiss such warn¬ 
ings [11, 24, 108, 135]. For the defenses we present in Chapter 7, we assume that attackers 
do not have access to the target site’s server machines or any secrets, such as private keys, 
contained thereon. 



16 


Chapter 3 

Conditioned-safe ceremonies 


In this chapter, we propose design principles for conditioned-safe ceremonies. At the 
high level, a conditioned-safe ceremony is one that deliberately conditions users to only 
perform safe actions during social engineering attacks and reflexively respond in ways that 
thwart attacks. Before discussing conditioned-safe ceremonies in detail, we first examine 
psychological tendencies that make users vulnerable to social engineering attacks on the 
Web and model user behavior during attacks with seminal psychological frameworks for 
human performance and error. In Chapter 4, apply our design principles to develop a 
conditioned-safe registration ceremony based on email. 

3.1 Why users are vulnerable 

Over the last decade, the success of Web based social engineering attacks, e.g., phish¬ 
ing, has created a multi-million dollar underground economy, largely by exploiting weak¬ 
nesses in password authentication and other Web authentication ceremonies [33, 117]. Al¬ 
though it is tempting to blame the success of these attacks on the ignorance of users, re¬ 
searchers have offered an alternative explanation: computer security mechanisms such as 
passwords and browser security indicators are poorly suited for human use [2, 22, 24, 25, 
46, 108, 129], 
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3.1.1 Automatic decision making strategies are exploitable 

A recurring theme in the psychological literature is that humans tend to vastly prefer 
rule-based decision making over more tedious analytical approaches [99, 100]. The the¬ 
ory of rule-based decision making is based on psychological studies that suggest humans 
tend to learn and aggressively apply problem-solving rules of the form “if (, situation ) then 
( action)” for frequently encountered situations. When a user encounters a problem in a 
task, she matches the most prominent cues in the environment with the calling conditions 
of previously learned rules to find most appropriate one to apply. 

Although rule-based decision making helps us navigate the minutia of our daily lives 
and reserve our time and energy for tasks requiring more detailed analysis, adversaries 
can exploit rule-based decision making in social engineering attacks [16]. Human relia¬ 
bility expert James Reason observed that frequently used rules, i.e., strong rules, may be 
“misapplied in environment conditions that share some common features with the appro¬ 
priate states, but also possess elements demanding a different set of actions [100].’’ In other 
words, a rule which has been frequently useful in the past can become a strong-but-wrong 
rule when the situational cues change subtly. This helps explains why phishing attacks 
have been so successful. Since a wide range of Web sites require a user to log in before she 
can do something interesting, many users have developed a rule of the form “if (login form) 
then (enter username/password)” and will aggressively apply it when they encounter login 
prompts on Web pages which on the surface appear familiar, legitimate, or trustworthy. We 
discuss rule-based decision making further in Section 3.2. 

3.1.2 Web browser security indicators condition users to satisfice 

A frequently recommended defense against phishing attacks is for a user to verify a Web 
page’s domain and SSL certificate before entering her password on that page; otherwise, 
she might inadvertently reveal her credentials to an attacker. However, research has shown 
that users often omit these checks [24, 25, 35, 37, 49, 56, 108, 130, 135]. Although some 
users ignore these indicators because they do not understand them, a more fundamental 
problem is that browser security indicators condition users to satisfice. 

Satisficing is a decision-making strategy which means “to accept a choice or judgment 
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as one that is good enough”, i.e., one that both satisfies and suffices [101]. Checking se¬ 
curity indicators is easy to skip because it distracts the user from her primary focus, and 
there are rarely any immediate visible consequences for skipping these checks or rewards 
for making them. Since the vast majority of a user’s login attempts are probably not under 
attack (or at least do not obviously appear to be under attack), routinely skipping secu¬ 
rity checks and ignoring warnings seems deceivingly acceptable. Over time, users learn 
to quickly and instinctively perform a security task’s required actions (e.g., entering their 
passwords) and optimize out the optional actions (e.g., checking security indicators, re¬ 
sponding to security warnings). Once a user has become conditioned into a satisficed be¬ 
havior, psychologists have found it is difficult for her to change it, even if she recognizes 
overwhelming evidence that her behavior is wrong [16]. 

3.1.3 Users are not good at recognizing attacks 

A recurring theme in the field of human reliability and error is that users often have 
difficulty in recognizing risky or dangerous situations, and as a result, users may be less 
vigilant of their choices in these situations than they should. For example, in a review of 
100 maritime shipping accidents, Wagenaar and Groeneweg concluded: “Accidents do not 
occur because people gamble and lose, they occur because people do not believe that the 
accident that is about to occur is at all possible [126].” This suggests that we cannot rely 
on users’ abilities to detect social engineering attacks and respond appropriately either, and 
we must design defenses accordingly. 

Several human tendencies contribute to this phenomenon. Psychological studies have 
shown that people are generally poor at evaluating risk and tend to believe they are less vul¬ 
nerable to risks than others [112]. For example, many people believe they are better than 
average drivers and they will live longer than the average life expectancy. It stands to reason 
these tendencies also influence user behavior on the Internet, leading users to believe they 
are less likely to be the targets of attacks. One might respond that security mechanisms, 
such as warnings, can raise awareness and help users recognize attacks. However, not only 
are security tasks rarely users' primary goal, but they often inhibit users from completing 
tasks. Security mechanisms often present users with two options: 1) terminate, or 2) pro- 
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ceed insecurely. Task failure is guaranteed by choosing option 1, but the probability of a 
security compromise is unclear with option 2. Since task failure is unsatisfying, there is 
a strong incentive to choose option 2. Decades of buggy software have conditioned users 
to expect errors, failures, and other incomprehensible system behavior, particularly with 
hastily developed and continually updated Web applications. Users routinely encounter 
warnings and errors messages, but rarely experience any immediate negative consequences 
for dismissing them, even during a real attack. This creates the (accurate) impression that 
false positives are the norm, and actual attacks are rare. 

Unfortunately, frequent false alarms and errors may be molding recklessly obstinate 
users. Studies suggest that once someone adopts a particular belief, e.g., “it’s unlikely I 
would be the victim of an attack”, it is hard to change her mind, even in the face of strong 
disconhrmatory evidence [105]. In the psychology community, this is known as belief 
perseverance. Belief perseverance is often sustained by confirmation bias [89], a human 
tendency to overweight confirming evidence for one’s belief and underweight disconfirm- 
ing evidence. Researchers have observed evidence of confirmation bias in users’ strategies 
for determining a Web site’s trustworthiness. Studies suggest users tend to rely more on a 
Web site’s look-and-feel, logos, their past experience at site, and personalized information 
to determine trustworthiness than standard browser security indicators such as the lock icon 
or URL bar [24, 25, 32, 59, 135]. Since adversaries can easily spoof look-and-feel, logos, 
and personalized information, these strategies are not likely to falsify her hypothesis, only 
confirm it. 

Underweighting disconfirming evidence often manifests itself as “explaining away” 
the inconsistencies [133]. Studies suggest that users will readily “explain away” attack 
instructions, broken images, strange URLs, and security warnings as errors rather than 
potential attack cues [24, 25, 28, 135]. Since users have been forced to develop strategies 
to work around computer system errors, it’s understandably tempting for user to interpret 
potential attack indicators as errors. Instead of having to terminate her task in response to 
an attack, by reclassifying the attack as an error, she can instead try to “work around” it by 
complying with the attacker’s “helpful” instructions. 
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3.2 Modeling user decision making during attacks 

3.2.1 Skill, rule, and knowledge based performance 

To help understand users' decision making processes during social engineering attacks, 
we look to Jen Rasmussen’s seminal skill-rule-knowledge classification of human perfor¬ 
mance [99]. Rasmussen’s skill-rule-knowledge framework has been used within the sys¬ 
tems reliability community to model human error [100]. Rasmussen's framework models 
human performance at three levels: skill-based, rule-based, and knowledge-based. Ras¬ 
mussen distinguishes performance levels based on whether the user is actively engaged in 
problem solving. The rule-based and knowledge-based levels are typically associated with 
problem solving activities, while the skill-based level is not. Behavior at the skill-based 
level generally encompasses routine and highly practiced sensorimotor actions which re¬ 
quire little conscious control after initiation. Although users may employ skill-based ac¬ 
tions to achieve local goals in problem solving tasks, skills are primarily invoked during 
routine and nonproblematic tasks in familiar situations. For example, consider the task of 
browsing a Web site. To navigate Web pages, a user commonly employs skill-based actions 
such as clicking URLs and scrolling until she encounters an unexpected problem or new 
situation. 

After identifying the existence of a problem, Rasumessen conjectures that users enter 
the rule-based level. The rule-based level applies to common, familiar problems for which 
the user has previously learned a successful solution. A recurring theme in the psycholog¬ 
ical literature is that humans are aggressive pattern matchers, and when they encounter a 
problem, they tend to search for a solution by matching prominent cues in the current sit¬ 
uation against learned problem-solving rules of the form “if ( situation ) then (act/on)”. For 
example, since many interesting Web sites require authentication, this framework suggests 
users will quickly learn a rule of the form “if (login form) then (enter username/password)” 
and aggressively apply it when they encounter login prompts on Web pages. Solutions 
generated at the rule-based level are analogous to Cialdini’s click-whirr responses. 

The knowledge-based level only comes into play during novel situations where a solu¬ 
tion must be planned online, using conscious analytical processes and stored knowledge. 
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Operating at the knowledge-based level is expensive; it often requires the user to inter¬ 
rupt her primary task, switch her attention, analyze the situation, form a mental model, 
and hypothesize and evaluate solutions. While browsing the Web, a user might enter the 
knowledge-based level if after attempting to log in, she receives an incomprehensible error 
message or her browser crashes. Psychological studies suggest that human strongly prefer 
to pattern match and will only resort to detailed analysis at the knowledge-based level after 
exhausting all potential prepackaged solutions at the rule-based level. Users are often so 
overwhelmed at the knowledge-based level that problem solving resorts to “sticking their 
head in the data stream until a recognizable pattern appears”, allowing them revert back to 
the rule-based level [100]. 

3.2.2 Human error and the Generic Error-Modeling System (GEMS) 

The Generic Error-Modeling System (GEMS) is a framework developed by James Rea¬ 
son for understanding human error during decision making and problem solving [100]. 
GEMS relies heavily on Jen Rasmussen’s skill-rule-knowledge framework and categorizes 
errors according to the performance level at which they occur. Errors at skill-based level 
are called slips, and errors at the rule-based and knowledge-based levels are called mis¬ 
takes. Slips are execution failures, where the user decides on an action, but the resulting 
action is not what was intended. For example, a user commits a slip if she decides to left 
click the mouse to follow a HTML link, but accidentally right clicks instead. In contrast, 
mistakes are planning failures, where the user decides on a course of action, but the action 
is not appropriate. For example, if a user has different passwords for different Web sites, 
she commits a mistake if she enters the wrong one. 

A frequent cause of rule-based mistakes is the misapplication of a good rule, i.e., a rule 
which has proven itself useful in the past. For example, “if (legitimate looking login form) 
then (enter username/password)” is usually a good rule in non-adversarial situations; its 
application allows a user to quickly authenticate herself and continue her primary task with 
minimal interruption. This rule is also strong for many users, meaning that it easily comes 
to mind due to repeated use. “Gambling” with strong rules makes sense if applying the 
rule frequently results in a winning solution. However, since users often match the calling 
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conditions of rules with the most prominent cues in the environment. Reason observes that 
strong, good rules may be “misapplied in environment conditions that share some common 
features with the appropriate states, but also possess elements demanding a different set of 
actions” [100]. A rule which has been frequently useful in the past can become a strong- 
but-wrong rule when the situational cues change subtly. As Taylor and Crocker put it [116], 
“Like all gamblers, cognitive gamblers sometime lose.” Phishers can create a severely 
rigged game for cognitive gamblers. Phishers provide situational cues which closely match 
the calling conditions “if (legitimate looking login form) then (enter usemame/password)”, 
and applying this rule in the presence of an attacker results in an security failure, i.e., the 
compromise of the user’s password. 

Alternatively, a rule-based mistake may also occur if the user applies a bad rule. Bad 
rules are exactly that, but often do not cause accidents unless users apply them in risky 
situations. Reason terms this type of bad rule an inadvisable rule. Inadvisable rules may 
be perfectly adequate to achieve goals under normal conditions, but under risky conditions 
they may cause accidents. Reason explains that inadvisable rules “arise when an individual 
or organization is required to satisfy discrepant goals, among which the maintenance of 
safety is often a very feeble contender.” Ignoring security warnings is excellent example 
of an inadvisable rule that many users have adopted. Carefully reading warnings is like 
“security maintenance” that users will quickly satisfice away if it becomes too burdensome. 

At the knowledge-based level, mistakes occur because the user is essentially flounder¬ 
ing. The fact that the user has reached the knowledge-based level means the situation is 
not immediately familiar to her and she has already exhausted any immediately applicable 
rules. The user must either form a mental model on-the-fly or reformulate the problem 
as a more familiar one. These tactics are highly prone to error, and confirmation bias 
and the tendency to explain away inconsistencies befog the decision making process. A 
faulty mental model or inappropriate problem reformulation may lead to the application of 
strong-but-wrong and inadvisable rules, further increasing the likelihood of errors. 
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3.2.3 Applying GEMS to user behavior during social engineering at¬ 
tacks 

Although GEMS was originally conceived for modeling human errors which lead to ac¬ 
cidents, researchers have observed GEMS is also useful for modeling human errors which 
lead to security failures [13, 21]. In particular, we argue that GEMS is useful for under¬ 
standing human behavior during social engineering attacks on the Internet. By casting 
social engineering attacks in the context of GEMS, we can hopefully apply lessons learned 
by human reliability specialists to help users resist these attacks. 

Generally, for an accident to occur, unsafe acts must combine with risky or unusual 
environmental conditions. During a social engineering attack, the presence of an adversary 
is the risky condition, and the adversary’s goal is to intentionally trigger human errors, 
typically through a combination of environmental manipulation and active encouragement. 
Current Web ceremonies such as password authentication create a lose-lose situation for 
users. A common modus operandi of adversaries is to mimic trusted Web sites, and if 
the charade succeeds, an adversary gains the opportunity to exploit strong-but-wrong and 
inadvisable rules. Alternatively, if the user is suspicious, she is all but forced to proceed 
at the knowledge-based level to ultimately decide if the site is trustworthy. Attacks are an 
exceptional situation for most users, and users are ill-equipped to address them. Users are 
most likely to make mistakes at the knowledge-based level, and studies suggest that many 
users have erroneous mental models of Web security mechanisms and have a wide variety 
of faulty defense strategies for phishing attacks [24, 25, 135]. For example, a user in one 
phishing study believed that she could enter her usemame/password to authenticate a Web 
site, assuming that only the legitimate site would be able to respond correctly [24]. 

In summary, we hypothesize that social engineering attacks on the Internet succeed 
mostly due a combination of two tactics: 1) tricking users into applying strong-but-wrong 
and inadvisable rules, and 2) forcing users to make security decisions at the knowledge- 
based level, where they are most likely to make mistakes. To help users resist social engi¬ 
neering attacks on the Internet, we argue that ceremonies should condition users to apply 
rules that serve them well and help them thwart attacks. In the next section, we introduce 
the notion of a conditioned-safe ceremony, an approach for designing ceremonies that takes 
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these observations into consideration. 


3.3 Conditioned-safe ceremonies 

One natural response to the weaknesses of current Web ceremonies such as challenge 
questions and passwords is to design ceremonies which try to eliminate user condition¬ 
ing, click-whirr responses, and rule-based decision making. This approach is problematic. 
Rule-based decision making is fundamental to human behavior: it helps us complete rou¬ 
tine tasks quickly and easily. Users may be willing to invest extra time and effort to learn a 
new security mechanism, but if they cannot learn how to use it efficiently, they will become 
frustrated and disable or circumvent the offending mechanism [41, 46, 141]. Some degree 
of conditioning may be necessary for the psychological acceptance of security mechanisms 
by users. 

Since users will tend to adopt rules for completing a ceremony that minimize conscious 
effort, we should not fight users’ tendencies to use rule-based decision making, but take 
advantage of these tendencies to help users resist social engineering attacks. We should 
prudently design ceremonies to condition rules that benefit security rather than undermine 
it. Towards achieving this goal, we introduce the notion of a conditioned-safe ceremony. 
A conditioned-safe ceremony is one that deliberately conditions users to reflexively act 
in ways that protect them from attacks. We propose two design principles for building 
conditioned-safe ceremonies: 

• Conditioned-safe ceremonies should only condition safe rules, i.e., rules that are 
harmless to apply in the presence of an adversary. 

• Conditioned-safe ceremonies should condition at least one immunizing rule, i.e., a 
rule which when applied during an attack causes the attack to fail. We discuss im¬ 
munizing rules further in Section 3.3.1. 

These principles also have important consequences on what conditioned-safe ceremonies 
should not do: 
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• Conditioned-safe ceremonies should not condition rules that require users to decide 
whether it is safe to apply them. Since many users are unreliable at recognizing 
risky situations, users should not need to refrain from conditioned behavior to resist 
attacks. 

• Conditioned-safe ceremonies should not assume users will reliably perform actions 
that: 1) the ceremony has not conditioned her to perform, or 2) are voluntary. Satis¬ 
ficing users will learn to omit optional and voluntary actions, so ceremonies should 
not rely upon users to perform such actions. 

For example, a ceremony should not condition the rule “if (legitimate looking login form) 
then (enter usemame/password)” because it causes a security failure when applied in the 
presence of an adversary. To determine if it is safe to apply this rule, a user must first verify 
the URL bar, the site’s SSL certificate, and other security indicators. Burdening users with 
these decisions is unsatisfactory. Ideally, in a conditioned-safe ceremony, a user should be 
able to resist an attack even if she has no idea she is at risk and performs the same actions 
as she usually performs under benign conditions. 

However, user behavior is unpredictable and an adversary may try to trick users into 
deviating from their normal, conditioned behavior in a way that causes a security failure. 
Conditioned-safe ceremonies need safeguards to resist these attacks. In the human reli¬ 
ability community, designers often introduce constraints called forcing functions to help 
prevent errors in safety-critical environments. We argue that forcing functions can also be 
useful for conditioned-safe ceremonies, and we discuss them further in the next section. 

3.3.1 Forcing functions 

A forcing function is a type of behavior-shaping constraint designed to prevent human 
error [90]. Forcing functions usually work by preventing a user from progressing in her 
task until she performs a particular action whose omission would result in a failure or 
accident. Because users must take this action during every instance of the task, the forcing 
function conditions users to always perform this action. With an effective forcing function, 
after a user performs the function's associated action, many mistakes become difficult or 
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impossible to make. For example, consider the error of locking your keys in your residence. 
A potential forcing function in this situation is a door that can only be locked from the 
outside, keys in hand. This trains you to take your keys with you whenever you leave 
home, making it less likely you will be locked out in the future. 

Forcing functions often have two benefits: 1) they help prevent errors of omission, 
where a user skips an important, protective step in a task, and 2) they condition correct, 
safe behavior, since users cannot normally proceed otherwise. To be effective, the cognitive 
and physical effort required to comply with a forcing function must be less than the effort 
required to circumvent it. Otherwise, users may routinely attempt to circumvent the forcing 
function, diminishing its benefits. 

Since forcing functions have been useful for preventing errors in safety-critical environ¬ 
ments, we hypothesize they can also help prevent errors during social engineering attacks. 
However, designing forcing functions that resist social engineering attacks is challenging. 
In conventional safety-critical environments, the risk elements rarely try to intentionally 
subvert protection mechanisms and cause errors. Designing electrical safety equipment 
would be a much trickier business if electricity had malicious intent. Also, deployability 
considerations for many ceremonies, e.g., no custom hardware, often require forcing func¬ 
tions to be implemented entirely in software. Software environments afford attackers many 
opportunities for mimicry. 

One previous application of software-based forcing functions in computer security is 
the concept of a secure attention key (SAK). A SAK is a mandatory special key combina¬ 
tion users must type before they can take a security-critical action, e.g., submitting their 
password. On Window NT systems, users must type Control-Alt-Delete to get a login 
prompt. The SAK diverts control to the OS kernel, foiling any user-level spoofed login 
prompts. Since typing the SAK is mandatory, the hope is that users will learn to always 
enter the SAK before submitting their password. 

Unfortunately, a simple attack against many SAKs is to induce an error of omission. 
On Windows NT systems, an adversary can display a spoofed login prompt and hope users 
skip the SAK before entering their passwords. This attack creates a conflict between two 
click-whirr responses: SAK systems condition users to first type the SAK, but all password 
systems condition users to enter their passwords when they see a login form. Whether the 
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attack succeeds depends on which click-whirr response is stronger in a particular user. 

Since social engineering attacks can often misrepresent the state of a system and create 
the illusion that a forcing function has already been activated or disabled, ceremonies which 
fail solely due to errors of omission are suboptimal. Errors of omission are easy to make 
and hard to detect, even during routine tasks. Research suggests that users frequently do not 
notice when they have omitted routine procedural steps [5], and omission errors represent 
one of the most common causes of human performance problems [98]. 

To resist social engineering attacks, we argue that conditioned-safe ceremonies need 
defense in depth. Designers should build conditioned-safe ceremonies that have two levels 
of protection: an attack should fail unless a user both omits the conditioned action required 
by a forcing function and makes an error of commission. We consider an error of commis¬ 
sion to be an anomalous user action not normally used in the ceremony. If the user omits the 
action required by the forcing function, but does not otherwise deviate from the ceremony, 
an attack should fail. Likewise, if the user performs the required action, but then makes an 
error of commission, the attack should also fail. With this approach, the action conditioned 
by the forcing function acquires an immunizing quality, since after a user performs this 
action, subsequent errors of commission will not compromise the ceremony. 

We emphasize that the conditioned action required by the forcing function must be 
easy for users to perform; in particular, it should easier to perform than any unsafe error 
of commission. Since humans have been conditioned to work around buggy software, a 
user may willingly make an effortless error of commission if she feels it will complete the 
security task and allow her to continue with her primary task. 

3.3.2 Analysis and discussion 

Although a designer can choose the rules conditioned by a ceremony, an attacker can 
affect which rules a user chooses to apply by manipulating the environmental stimuli. Re¬ 
search by psychologists and human reliability specialists suggests that users mainly rely 
on two processes to determine the most appropriate rule to apply in a given situation: 
similarity-matching and frequency-gambling [100]. With similarity-matching, a user com¬ 
pares the situation’s environmental cues against cues contained in the calling conditions of 
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previously learned rules. If she finds a unique match, she performs the associated action. If 
the environmental cues are underspecified and partially match several rules, she will tend 
to “gamble” in favor of the useful, high frequency candidates, i.e., the “good” rules which 
have been most frequently applied in the past. 

These tendencies suggest that conditioned-safe ceremonies will better resist the cur¬ 
rently successful attack strategy of blatantly initiating a ceremony with the victim and 
presenting familiar environmental cues, e.g., spoofing a trusted Web site. Since a forc¬ 
ing function requires a user to perform the immunizing action every time (whether under 
attack or not), the forcing function will condition a high-frequency, “good” rule (namely, 
perform the immunizing action) that is likely to be routinely applied in the future - even 
when under attack. Mimicking a conditioned-safe ceremony becomes less advantageous to 
an adversary; if a user recognizes she is participating in the ceremony, she will tend to per¬ 
form the conditioned, immunizing action, which thwarts attacks. This presents an attacker 
two options: 1) obviously initiate the ceremony and try to induce an error of commission 
before the user performs the immunizing action, or 2) surreptitiously initiate the ceremony 
and try to induce an error of commission without the user realizing she is participating in 
the ceremony. 

If attackers resort to the first option, adversaries must prevent the human tendency to 
use rule-based decision making, rather than encourage it, as current attacks do. This creates 
a disadvantage for adversaries; preventing human tendencies is often difficult. If attackers 
resort to the second option, we hope adversaries will need to present unfamiliar situations to 
prevent users from recognizing the ceremony. Otherwise, users will tend to react with con¬ 
ditioned responses, i.e., apply safe rules and perform immunizing actions. This approach 
also disadvantages adversaries. Unfamiliar situations require additional cognitive effort to 
analyze and may cause feelings of suspicion and discomfort. User often reject unfamiliar 
experiences in favor of more familiar ones. For example, studies suggest that some users 
distrust phishing warnings because the familiar experience presented by the adversary ap¬ 
pears more trustworthy [28, 135]. Conditioned-safe ceremonies turn the tables and force 
adversaries to be the ones required to present an awkward and unfamiliar experience. 
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3.3.3 Limitations 

We acknowledge conditioned-safe ceremonies have their limits. Adversaries may try 
to convince users to disable protective mechanisms or take actions outside the scope of a 
ceremony which violate certain security assumptions. For example, with the configura¬ 
tion of many current computer systems, if a user chooses to install malware at any point, 
most ceremonies will be compromised. However, if we can design ceremonies that are so 
unproductive to attack directly that adversaries must resort to convincing users to install 
malware, it would be a tremendous step forward. 
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Chapter 4 

Registration ceremonies 


Since users may use more than one computer, machine authentication systems must 
have a registration ceremony to authorize and set authentication cookies on multiple ma¬ 
chines. Many machine authentication systems currently deployed by financial Web sites 
use challenge questions in their registration ceremonies [9, 52, 123]. In this chapter, we 
describe weaknesses of registration ceremonies based on challenge questions and present a 
conditioned-safe registration ceremony we developed based on email. 

4.1 Current practice: Challenge questions 

A challenge question is a user-specific question which an adversary is unlikely to be 
able to guess an answer, e.g., “What is the name of your favorite teacher?” [30, 65]. When 
a user creates her account, she provides the answers to one or more challenge questions, and 
when she attempts to log in from an unregistered computer, the site prompts her to answer 
these questions. If the answers are correct, then the site sets a persistent authentication 
cookie on the user’s computer. For future logins from that computer, the user only needs to 
enter her password. 

Challenge questions are vulnerable to an active man-in-the-middle (MITM) attacker 
spoofing the login page of the target Web site [113, 144]. 1 When a user attempts to login 

1 Challenge questions also have other vulnerabilities [96, 118] which are not directly relevant to our work, 
e.g., the answers may be relatively easy to guess for many users. 
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via the spoofed page, the attacker forwards the user’s login credentials to the legitimate 
Web site. Since the attacker is indistinguishable from the actual user logging from an 
unregistered machine, the Web site responds with challenge questions for the user. The at¬ 
tacker displays these questions to the user. After the user provides her answers, the attacker 
forwards them to the Web site and receives an authentication cookie for the user’s account. 

Challenge question based registration is vulnerable because, like password authentica¬ 
tion, it disregards human tendencies and conditions users to fall for attacks. A user is most 
likely to resist an attack against her challenge questions if she recognizes the threat and 
refrains from the click-whirr response of providing her answers. However, since the at¬ 
tacker’s registration request is indistinguishable from the Web site’s legitimate registration 
requests, detecting attacks is non-trivial for many users. Users must actively and carefully 
check browser security indicators, e.g., the URL bar and SSL certificate, to detect spoofing 
attacks. Many users misinterpret these indicators, and satisficing users often ignore them. 

In theory it is possible a user who has previously registered her machine and under¬ 
stands how registration works may be suspicious of the attacker’s “spurious” registration 
request. However, users may not suspect they are under attack if there is any other rea¬ 
sonable explanation for the spurious request. If the user views the attacker’s request as an 
error, e.g., her computer was accidentally “forgotten”, the natural “fix” is for her to answer 
her challenge questions. 

Regardless, this assumption about users’ mental models is probably too strong. Since 
many users misunderstand browser security indicators, it is reasonable to expect many 
users will misunderstand registration procedures as well. A probable misinterpretation of 
registration procedures is that the Web site randomly asks challenge questions from time to 
time to verify the user’s identity, and an attacker’s request is hardly suspicious within this 
mental model. 

Registration based on challenge questions threatens to undermine the promise of ma¬ 
chine authentication. Since users who are vulnerable to phishing attacks against their pass¬ 
words will probably also be vulnerable to phishing attacks against their challenge ques¬ 
tions, a registration ceremony using challenge questions is hardly more secure than using 
passwords alone. We need better registration ceremonies to realize the benefits of machine 
authentication. 
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4.2 A conditioned-safe registration ceremony using email 

In this section, we describe a conditioned-safe registration ceremony for machine au¬ 
thentication using email. When a user attempts to log in from an unregistered computer, 
the Web site sends her an email containing a single-use HTTPS URL with an unpredictable 
component, for example: 

https://www.xyz.com/reg.php?url_id=r 

where r is a 160 bit random number generated by the Web site. 2 We call this URL a 
registration link. The email includes instructions to click on the link. The Web site stores r 
in a database, along with the associated user ID, an expiration time, and validity bit. When 
the user clicks on the registration link, if r is still valid and has not expired, the Web site 
sets a persistent authentication cookie on the user’s computer and invalidates r. A user 
only needs to complete this ceremony once at each computer. For subsequent logins, she 
only needs to complete any supplementary login procedures, e.g., enter her username and 
password. Several researchers have proposed using email in a similar way to help initialize 
authentication credentials [3, 8, 38, 44, 122]. 

4.2.1 Security analysis. 

Against the phishing threat model, we argue email registration follows the principles 
of a conditioned-safe ceremony we propose in Section 3.3. The phisher can solicit the 
user’s login name and password, but since the phisher’s computer is unregistered, the site 
will not allow it to access the user’s account without submitting a valid registration link. 
The attacker can trick the Web site to send the user a registration link, but to compromise 
the ceremony, an attacker must steal and use a registration link before the user submits it 
herself. 3 

2 We assume the user has previously given the Web site her email address, e.g., during the account creation 
process. 

3 We do not consider attacks which enable adversaries to steal users’ authentication cookies after they have 
been set, e.g., cross-site scripting attacks or malware. This problem is orthogonal to registration and requires 
a different solution. 
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The registration link acts as forcing function. Under normal conditions, a user must 
click on the link to proceed. Although there may be other ways of submitting the link, 
e.g., by copying and pasting it in the URL bar, clicking generally requires less effort, and 
sites can embed the URL of the link in an HTML element to make the alternatives more 
difficult. Also, clicking on the registration link is an immunizing action; after the Web site 
invalidates the link, it is useless to an attacker. 

Email based registration has defense in depth. To compromise the ceremony an attacker 
must 1) prevent the user from clicking on the link (i.e., omit the forcing function action), 
and 2) trick the user into revealing the link (i.e., make an error of commission). One 
possible attack strategy would be to inform the user that she must register her computer, 
but due to “technical problems” she should not click on the link and instead give the link 
to the attacker. We identify two compelling and straightforward attacks of this kind: 1) 
ask the user to copy and paste the registration link into a text box, or 2) ask the user to 
forward the registration email to an address with a similar domain name as the target site. 
If a user does not notice the attacker’s instructions and believes she is participating in the 
“normal” registration ceremony, we hypothesize she will likely resist these attacks. Email 
registration conditions users to click on the registration link, and if she clicks the link, she 
will resist the attack. 

Alternatively, if the user notices the attacker’s instructions to deviate from the cere¬ 
mony, she will be safe as long as she clicks on the link before doing anything else. Since: 
1) the Web site has conditioned the user to click on the registration link; 2) the credible 
repercussions of clicking on link are probably limited; and 3) clicking on the registration 
link is arguably at least as easy as complying with the instructions, the theory of rule-based 
decision making suggests that users will first tend to try clicking on the registration link 
before complying with the adversary’s instructions. 

The key question is the strength of users’ tendencies to click the registration link rather 
than comply with the adversary’s instructions. To help answer this question, we conducted 
a user study to estimate how well email registration helps users resist social engineering 
attacks against it. In the next chapter, we describe this study. 
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4.2.2 Implications and limitations of email based registration 

One might argue that ceremonies that require users to click on email links will train 
users to click on phishing links and undermine some anti-phishing efforts which caution 
users to never click on links in email. However, we argue that relying on users to never 
click on email links is unrealistic. Sending and clicking on links in email is often useful for 
users, and many password reset and recovery ceremonies currently require users to click 
on an email link [38]. Some phishing studies suggest that many users regularly click on 
email links and employ a wide variety of link clicking strategies based on the current task, 
apparent source of the email, and other contextual cues [24, 26], It would be a significant 
challenge to eliminate these practices. We argue that more comprehensive defenses which 
assume users will click on some email links are more likely to be effective. 

Another potential criticism is that email based registration shifts many of the security 
and usability burdens onto email systems. The security of email systems relies on the 
security of email servers and users’ email passwords. This raises several concerns [38]: 

• A user might use a weak email password or use the same password for all her ac¬ 
counts. 

• Some email providers use weak password reset and recovery mechanisms, such as 
challenge questions, which may be vulnerable to social engineering and inference 
attacks. 

• Users may view their email accounts as less sensitive than their financial accounts 
and fail to adequately protect their email passwords. In our study presented in the 
next chapter, many users viewed the security of their email accounts as having the 
same level of importance as their accounts at social networking sites, but below their 
accounts at financial sites. 

• Email is often sent over unencrypted connections, and POP and IMAP servers often 
accept passwords sent over unencrypted connections. 

• Employees at businesses or ISPs might have access to their users’ email. 

• Several users might share a single email account. 
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• Email delivery is sometimes delayed. 

• Spam filters may block legitimate messages. 

Although the widespread use of email for password recovery and reset suggests that these 
issues may be manageable, we should not ignore them. Ideally, we should explore more 
secure and reliable messaging alternatives for security critical applications. One potential 
direction is to send registration links to users’ mobile phones and develop software which 
enables easy transfer of the links to users’ computers. 
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Chapter 5 

A user study of registration ceremonies 


In this chapter, we describe a user study we conducted to compare the security of email 
registration to the security of registration using challenge questions. Our study simulated 
man-in-the-middle (MITM) social engineering attacks against users of each of the cere¬ 
monies. Our hypothesis is that email registration is significantly more resistant to MITM 
social engineering attacks than registration using challenge questions. 

5.1 Design challenges 

Ecological validity is crucial: our study must realistically simulate experiences users 
have in the real world. This raises a number of challenges: 

First, it is difficult to simulate the experience of risk for users without crossing ethical 
boundaries [58]. To address this, many experimenters employ role-playing, where users 
are asked to assume fictitious roles. However, role-playing participants may act differently 
than they would in the real world. If users feel that nothing is at stake and there are no con¬ 
sequences to their actions, they may take risks that they would avoid if their own personal 
information was at stake. 

Second, we must limit the effect of demand characteristics. Demand characteristics 
refer to cues which cause participants to try to guess the study’s purpose and change their 
behavior in some way, perhaps unintentionally. For example, if they agree with the hypoth¬ 
esis of the study, they may change their behavior in a way which tries to confirm it. Since 
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security is often not users’ primary goal, demand characteristics are particularly challeng¬ 
ing for security studies. An experiment which intentionally or unintentionally influences 
users to pay undue attention to the security aspects of the tasks will reduce its ecological 
validity. 

Third, we must minimize the impact of authority figures during the study. Researchers 
have shown that people have a tendency to obey authority figures and the presence of au¬ 
thority figures can cause study participants to display extreme behavior they would not 
normally engage in on their own. Classic examples of this are Milgram’s “shocking” ex¬ 
periment [81] and the Stanford prison experiment [48]. For security studies, this tendency 
may underestimate the strength of some defense mechanisms and overestimate the success 
rate of some attacks. For example, if we simulate a social engineering attack during the 
study, users may be more susceptible to adversarial suggestions because they misinterpret 
these to be part of the experimenter’s instructions. They may fear looking incompetent or 
stubborn if they do not follow the instructions correctly. This problem may be exacerbated 
if there is an experimenter lurking nearby. 

Fourth, we must identify an appropriate physical location for the experiment. The vast 
majority of previous security user studies simulating attacks have been conducted in a con¬ 
trolled laboratory environment. They are many advantages to a laboratory environment: 
the experimenter can control more variables, monitor more subtle user behavior, and de¬ 
brief and interview participants immediately upon completion, while the study is still fresh 
in their minds. But a laboratory environment also has some drawbacks for security stud¬ 
ies. Since laboratories often lack the distractions and noise of the real world, users may be 
more likely to notice subtle or exceptional events. Also, users may evaluate risk differently 
than they would in the real world. A user may view the laboratory environment as safer 
because they feel that the experimenter “wouldn’t let anything bad happen”. Lastly, a short 
laboratory study may implicitly pressure users to ignore an option they sometimes have in 
the real world - to put off a security decision indefinitely. 

It may be tempting to ignore some or all of these issues in a comparative study such 
as ours. Since the effects of these factors will be present in both the control group (i.e., 
challenge question users) and the treatment group (i.e, email registration users), then one 
might conclude that ignoring these factors would not hinder a valid, realistic comparison 
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between the two groups. 

This is a dangerous conclusion. It is not clear to what degree these issues affect various 
types of security-related mechanisms. In particular, there is no evidence that these issues 
have a similar magnitude of effect on challenge question users as on email registration 
users. Therefore, it is prudent to control these issues in our design as much as possible. 

5.2 Study design 

5.2.1 Overview 

Our study addressed the challenges we identified in Section 5.1 with two techniques: 
1) we did not use a laboratory, and 2) we employed deception to hide the study’s true 
purpose. We recruited users remotely, and during the consent process, we told users that 
our experiment aimed to determine how well individuals can predict high grossing movies. 
We told each user she will log into our Web site over a seven day period and make a 
prediction of what she thinks will be the top three highest grossing movies each day. Each 
user logged in from her “natural habitat”: from her own computer, from anywhere, and at 
any time she wished. We show a screenshot of our interface in Figure 5.1. 

Each user received $20 as base compensation, and we rewarded her up to an additional 
$3 per prediction depending on the accuracy of her predictions. We told each user that 
she must make seven predictions to complete the experiment, so the total maximum a user 
could receive is $41. 

We simulated the experience of risk by giving users password-protected accounts at our 
Web site and creating an illusion that money they “win” during the study was “banked” in 
these accounts. We paid users at the end of the study via PayPal and solicited each user’s 
PayPal email address at the beginning of the study. 1 To help suggest that there was a risk 
that the user’s compensation could be stolen if her account was hijacked, we provided an 
“account management” page which allowed the user to change the PayPal email address 
associated with her account. 

1 Although we did not verify each user’s PayPal account was valid at the start of the study, each user 
explicitly acknowledged she either had an account or was willing to get one. 
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Figure 5.1: User interface for making predictions at our study Web site. 

Although we told users they must make seven predictions to complete the study, after 
each user made her fifth prediction, we simulated a MITM social engineering attack against 
her the next time she logged in. After she entered her username and password, we redi¬ 
rected her to an “attack” server. We discuss the simulated attacks in Section 5.2.4. After the 
simulated attack, we debriefed each user about the true purpose of the study and requested 
her reconsent for the use of her data. 

5.2.2 Recruitment 

We recruited users through the Experimental Social Science Laboratory (Xlab) at UC 
Berkeley. The Xlab is an interdisciplinary facility that supports UC Berkeley investiga¬ 
tors in running behavioral and social science experiments. Members of the UC Berkeley 
community (i.e., students, staff, etc.) register with the Xlab over the Web and receive solic- 
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Group 

Size 

Registration 

method 

Attack description 

Warnings 

in email? 

1 

41 

Challenge questions 

Solicit answers 

N.A. 

2 

40 

Email 

Forward email to attacker 

/ 

3 

39 

Email 

Forward email to attacker 


4 

40 

Email 

Copy and paste link into text box 

/ 

5 

40 

Email 

Copy and paste link into text box 



Table 5.1: Summary of study groups. 


itations to participate in experiments via email. One limitation of this recruitment method 
is that our user pool was primarily composed of university students and staff and may 
not be representative of the general population. Our experiment used only native English 
speakers, and the subject pool included approximately 1950 eligible users. 

We contacted 225 randomly selected users in April 2008 through the Xlab. Interested 
users signed up through the Xlab’s system and received instructions to visit our study Web 
site. We did meet any of the users in person. 208 users signed up for our study, and we 
assigned them round-robin to 5 study groups. One group used challenge questions for 
registration and the other four groups used different variants of email registration links. We 
discuss the email registration groups further in Section 5.2.4. We excluded the results of 8 
users and give details in Section 5.3.1. We show a summary of the user groups and their 
sizes in Table 5.1. 

5.2.3 Registration procedures 

Each user created an account at our site, with a username and password. We also 
asked for the user’s email address and PayPal email address, if different. During a user’s 
first login attempt, we redirected her to the page shown in Figure 5.2 after she entered 
her username and password. This page informed her that she must register her computer 
before she can use it to access her account at our Web site. We also showed a user this 
page if she subsequently attempted to access our site from an unregistered computer. We 
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wished to encourage users to register only private computers, so the site mentioned that it 
is a generally a bad idea to register public computers and if the user was using one, then 
she should wait until later to register her private computer. However, we did nothing to 
prevent or detect a user registering a public computer, such as a library computer. If the 
user chose to register her computer, we redirected her to the registration page. If she was 
in the challenge question group, we prompted her to set up her challenge questions with 
the page shown in Figure 5.3. She selected two questions and provided answers. After 
confirming the answers, she entered her account and proceeded with her first prediction. 

If she was part of an email registration group, then she saw a page informing her that 
she had been sent a registration email and must click on the link labeled “Click on this 
secure link to register your computer”. After clicking on the link, she entered her account 
and made a prediction. We sent registration emails in HTML format, but also included a 
plain text alternative (using the multipart/alternative content type) for users who 
had HTML viewing disabled in their email clients. We embedded the same registration 
link in both parts, but included a distinguishing parameter in the link to record whether 
the user was presented with the HTML or plain text version of the email. We discuss how 
we used this information in Section 5.2.4. Screenshots of registration emails are shown in 
Figures 5.5(a) and 5.5(b). 

Both registration procedures set an HTTP cookie and a Flash Local Shared Object on 
the user’s computer to indicate the computer is registered. For subsequent login attempts, 
we first prompted the user for her username and password. If the username and password 
were valid, our server checked if the user’s computer was registered for that username. If 
she was logging in from a registered computer, then we redirected her to her account. If she 
was logging in from a computer we didn't recognize, then we prompted her to answer her 
challenge questions (Figure 5.4(a)) or sent her a new registration link to click on, depending 
on the user’s group. 
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Sorry, we do not recognize this computer. 


You must register this computer before you can use it to access your account. 

If you are using your private computer, then you can continue and register it. Please click: 

I am using a private computer, register it | 

If you are using a public computer, like one at a library or Internet cafe, please log in later from your 
private computer. If you register a public computer and someone steals your password, they may be 
able to log into your account from that computer. In this case, please click: 

I am not using a private computer, I will log in later | 


Figure 5.2: User interface for confirming registration. 



Figure 5.3: User interface for setting up challenge questions 
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(a) User interface for answering challenge questions. 


Please answer your challenge questions. 


To log in, please provide answers to the challenge questions you selected when you created your 
account. 


your challenge questions 


Question 1: In what city did you graduate high school? 
Answer 1: I 


Question 2: What is the name of your first pet? 
Answer 2: | 

Submit | Cancel | 


(b) Screenshot of the attack against challenge questions. 


Figure 5.4: Normal challenge questions interface vs. simulated attack instructions 
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Please register your computer. 


Hello JohnSmith, you made a request on April 28, 2008 18:55 to register a computer for your account at UCB 
Movie Predictions. 


Click on this secure link to register your computer. 


Warning! To protect the security of your account: 

• Never forward this email. 

• Never copy/paste any links or other information from this email. 

• The only safe action is to click on the link above. 

If you are ever asked to do anything other than click on the above link, it is possible someone is attacking your 
account. If you have any doubts, close your browser and visit UCB Movie Predictions at a later time. 

Please contact us at info@ucbmovieoredictions.com if you have any questions. Thanks! 


(a) HTML registration email with warnings. 


Please register your computer. 


Hello JohnSmith, you made a request on March 27, 2008 15:27 to register a computer for your account at UCB 
Movie Predictions. 


Click on this secure link to register your computer. 


Please contact us at info@ucbmoviepredictions.com if vou have any questions. Thanks! 


(b) HTML registration email without warnings. 


Figure 5.5: Registration emails 
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Please check your email at JohnSmith@gmail.com. 


To register this computer, please look for a registration email from UCB Movie Predictions at 

JohnSmith@gmail.com 

1, Because of problems with our email registration system, please DO NOT CLICK ON THE LINK in 
the email you receive. 

2 Instead, please forward the registration email to support@ucb-moviepredictions.com 
and then log in again 


I--1 

DO NOT CLICK ON THIS LINK IN YOUP EMAIL. Instead, please 
forward the email to support@ucb-moviepredictions.com 


Thanks! 


(a) Screenshot of the forwarding attack against email registration. 


Please check your email at JohnSmith@gmail.com. 


To register this computer, please look for a registration email from UCB Movie Predictions at 

JohnSmith@gmail.com 

1. Because of problems with our email registration system, please DO NOT CLICK ON THE LINK in 
the email you receive. 

2. Instead, please copy the link in the email into the box below. You can copy it by right clicking 
and selecting "Copy Link Location” or "Copy Shortcut". Then, enter it below with "Control-V". 

Submit | 


Click on this secure link to register your computer. 

- 1 - 

DO NOT CLICK ON THIS LINK IN YOUP EMAIL. 
Copy this link into the box above. 


Open 

Open m New Tab 
Open in New Window 
Save Target As... 
Print Target 


Copy Shortcut 


Thanks! 


(b) Screenshot of the cut and paste attack against email registration. 


Figure 5.6: Our simulated attacks against email registration, 
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5.2.4 Simulated attacks 

Challenge questions: Group 1 

For the challenge question group, the attack attempted to convince users to answer their 
challenge questions by presenting the page shown in Figure 5.4(b). This is essentially the 
same page users saw when they answered their challenge questions under “normal” condi¬ 
tions, but with the warning and informative text removed. 2 This attack: 1) is straightfor¬ 
ward, 2) closely mimics the legitimate registration process, and 3) was previously disclosed 
in the security community as a major weakness of challenge questions [113, 144], 

Email: Groups 2-5 

For the email groups, we simulated the two attacks we identified in Section 4.2: the 
copy and paste attack and the forwarding attack. The copy and paste attack asked the user 
to copy the registration link into a text box, and the forwarding attack asked the user to 
forward the registration email to an address with a similar domain name as our study site. 
We simulated the forwarding attack against groups 2 and 3, and simulated the cut and paste 
attack against groups 4 and 5. 

We chose these attacks because we believed they are the most compelling and straight¬ 
forward attacks that we could ethically implement. Another potentially effective attack 
would be to try to hijack each user’s email account, but we did not believe this attack was 
ethical. We leave other attacks as a subject for future work. 

For both attacks, the attack page first told the user that “because of problems with our 
email registration system” she should not click on the link in the email she received. For 
the copy and paste attack, the attack page presented a text box with a “submit” button and 
instructed the user to copy and paste the registration link into the box. For the forwarding 
attack, it instructed the user to forward the email to the attacker’s email address. We show 
screenshots of the attack pages in Figures 5.6(a) and 5.6(b). 

These attacks also presented pictorial versions of the instructions, with a screenshot 

2 Even if users select their challenge questions from a pool of possible questions, an attacker can eas¬ 
ily determine a particular user’s questions by relaying communications between the legitimate site and the 
user [113, 144], 
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of how the registration link appears in the email. To maximize the effectiveness of this 
picture, we gave the attacker the advantage of knowing the distribution of HTML and plain 
text registration emails previously viewed by the user during the study (see Section 5.2.3). 
The attack displayed the pictorial instructions corresponding to the majority; in case of a 
tie we displayed a screenshot of the HTML version. 

5.2.5 Warnings 

Some Web sites warn users about safe security practices, e.g., how to resist phishing 
attacks against challenge questions. Although these warnings are sometimes useful, they 
will likely be absent during an attack, when a user needs them the most. Email registration 
has the advantage of being able to include advisory information and contextual warnings in 
each registration email. To measure the effectiveness of these kinds of warnings, we subdi¬ 
vided the email groups into two groups: those who received warnings in registration emails 
(groups 2 and 4) and those who did not (groups 3 and 5). Everyone saw these warnings on 
legitimate registration pages. A screenshot of these warnings is shown in Figure 5.5(a). 3 
Group 1 users also received warnings about safe practices when answering their challenge 
questions, but we only showed group 1 users these warnings during legitimate registrations. 
Group 1 users never received warnings in email. 

5.2.6 Attack success metrics 

If a group 1 user answered her challenge questions correctly on the attack page, we 
considered the attack a success and ended the experiment. We assumed an attacker could 
distinguish between correct and incorrect answers (e.g., by relaying the user’s responses 
in real time to the legitimate site), so if a user entered an incorrect answer, the attacker 
prompted her again. 

If a group 2-5 user clicked on the registration link first, then we considered the attack a 
failure. 4 If the user forwarded the email or submitted the link first, then we considered the 

3 These warnings specifically warned against the attacks we simulated. Although in the real world it may 
not be feasible to concisely warn users against all the possible attacks, a site can certainly warn users against 
the most successful or common attacks they have observed in the past. 

4 These attacks actually simulated network level MITM attacks. Such attackers might be able to intercept 
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attack a success. Either way, we ended the experiment for the user. 

For all users, attempts to navigate to other parts of the site redirected the user back to 
the attack page. If the user resisted the attack for 30 minutes, then on her next login, the 
experiment ended and we considered the attack a failure. The attack pages for groups 1, 
4, and 5 contained a Javascript key logger, in case a user began to answer her challenge 
questions or entered the link, but then changed her mind and did not submit. If our key 
logger detected this, we considered the attack a success. 

5.2.7 Debriefing and exit survey 

After a user completed the study, we redirected her to a page that debriefed her about 
the true purpose of the experiment and explained the reasons for deception. The debriefing 
page explained the concept of machine authentication and the different ways of registering 
computers. We then obtained reconsent from each user. If a user reconsented, we redirected 
her to an exit survey. 

Our exit survey started with general demographic questions such as gender, age range, 
and occupational area. The second section of the survey collected information on the user’s 
general computing background, attitudes, and habits. The final section asked more specific 
questions about the user’s experiences during the study. The exit survey questions are 
shown in Appendix A. Users’ responses are in Appendix B. 

5.2.8 Ethics 

Our simulated attacks were ethical. The risk to users during the attacks was minimal. 
We only used data from users who explicitly reconsented after a debriefing on the true 
nature of the study. The study protocol described here was approved by the UC Berkeley’s 
Institutional Review Board on human experimentation. 

To protect users’ privacy, all connections to our Web site used SSL. We purchased a cer¬ 
tificate for our domain which is accepted by major Web browsers. In a real world attack, 

registration links and steal any registration tokens stored on the user’s computer. There are various propos¬ 
als that can help protect registration links and cookies against stronger adversaries [18, 54, 70], but we do 
not discuss the details here. Regardless, the results of this study are applicable to a wide variety of social 
engineering attacks, including phishing. 
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an attacker would most likely not be able to obtain a valid certificate for the target site. To 
avoid certificate warnings, an attacker would probably use HTTP rather than HTTPS to host 
the attack page. However, to protect users’ privacy, our simulated attack page used SSL. 
Since our hypothesis is that email registration is more secure than challenge questions, we 
had to ensure that our imperfect simulation did not bias the results against challenge ques¬ 
tions. Our solution was to maximize the benefits of SSL for the challenge question users 
and minimize the benefits of SSL for the email registration users. In particular, we con¬ 
servatively assumed that our simulated adversary attacking email registration had obtained 
a valid certificate for the target domain while our simulated adversary attacking challenge 
question based registration had not obtained a valid certificate. Group 2-5 users did not 
see certificate warnings during the attack, but group 1 users did. We implemented this by 
redirecting group 1 users to a different Apache instance (at port 8090) with a self-signed 
certificate, while group 2-5 users continued to use the original Apache instance in “attack 
mode”. This implies the “attack” domain shown in the URL bar for group 1 users included 
a port number, but the “attack” domain for group 2-5 users did not. 

5.3 Study results 

5.3.1 User demographics 

One email registration user did not complete the study, and one email registration user 
did not reconsent. Due to a misconfiguration, our server did not send registration emails 
to 6 users during the simulated attack. We excluded these users’ data from our results, 
leaving 200 users total. We summarize the demographics of the 200 non-excluded users in 
Tables 5.2 and 5.3, broken down by group number. 

56% of users self-reported themselves as female, 41% reported themselves as male, 
and 3% did not respond. Our users were mostly young: 91% reported themselves as 18-25 
years old and 89% reported themselves as undergraduate students. Among students, the 
mix of major areas was diverse. The largest group was physical sciences (i.e., chemistry, 
physics, biology, etc.), accounting for 25% of users, and the second largest was economics 
and business, accounting for 20% of users. Computer science and engineering accounted 



Status 


50 



Table 5.2: User demographics by group number. Some percentages may not add to 100% due to rounding. 
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for 1.5% and 7.5% of users, respectively. 

Most users reported using Windows (69%) and Mac OS (25%) as their primary oper¬ 
ating systems, and most users reported using Firefox (70%), Internet Explorer (11%), and 
Safari (10%) as their primary Web browsers. This differs significantly from recent statis¬ 
tics, which report Windows as having 91% market share and Internet Explorer as having 
72% market share [14]. 

78% of users reported using a Web browser at least 10 hours a week, and 70% of 
users reported they have conducted financial transactions online for at least a year. Types 
of online transactions reported include PayPal (55%), banking (80%), investing (12%), 
auctions (42%), and shopping (80%). 

5.3.2 Success of simulated attacks 

We summarize our results by group number in Table 5.4. Our attack succeeded against 
92.7% of challenge question users and 41.5% of email users. This difference is statistically 
significant (p < 0.001, Fisher’s exact test). The cut and paste attack was slightly more 
effective than the forwarding attack (47% vs. 40% with warnings, and 47% vs. 31% 
without warnings), but we did not find this difference significant (p = 0.65 with warnings, 
and p = 0.17 without warnings; Fisher’s exact test). We found no significant correlations 
between attack success and the demographics we reported in Section 5.3.1. In particular, 
we found no evidence that frequent browser use, previous experience with online financial 
transactions, or a technical undergraduate major area helped users resist our attacks. 

5.3.3 Efficacy of our warnings 

We found no evidence that including warnings in registration emails helped users resist 
our attack. To evaluate the effectiveness of our contextual email warnings, we compared the 
attack success rates of group 2 vs. group 3 (forwarding attacks, with and without warnings, 
resp.), and group 4 vs. group 5 (cut and paste attacks, with and without warnings, resp.). 
For the forwarding attack, 40% of group 2 users were vulnerable, and 31% of group 3 
users were vulnerable (p = 0.48 for Fisher’s exact test). For the cut and paste attack, 47% 
of users in both group 4 and group 5 were vulnerable (p — 1 for Fisher’s exact test). 
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Group 

Registration method 

Attack 

Warnings 

in email? 

Size 

Attack 

successful 

1 

Challenge questions 

Solicit answers 

N.A. 

41 

92.7% (38) 

2 

Email 

Forwarding 

/ 

40 

40.0% (16) 

3 

Email 

Forwarding 


39 

30.8% (12) 

4 

Email 

Cut and paste 

/ 

40 

47.5% (19) 

5 

Email 

Cut and paste 


40 

47.5% (19) 


Table 5.4: Success rates of our simulated attacks against registration procedures in 
our user study. Users in groups 2 and 4 received contextual warnings in registration emails 
against our simulated attacks, but users in groups 3 and 5 did not. 


During the exit survey, we showed each user a screenshot of the warning corresponding 
to her study group (Section 5.2.5) and asked her “Do you remember seeing the above 
warning at any point during the study?”, and if yes, “describe how it affected your decisions 
(if at all) while interacting with the study Web site.” Table 5.5 summarizes the number of 
yes/no responses. For group 2 and 4 users, who received warnings in registration emails, 
31% reported that they did not remember the warning. Among group 3 and 5 users, who 
only received warnings on the study Web page, 68% did not remember the warning. This 
difference is statistically significant (p < 0.001 for Fisher’s exact test). 66% of challenge 
question users also did not remember the warning. 

We found no evidence that users who recalled seeing our warnings were more likely 
to resist our attack. Of the 191 users responding to the warning recall question, 85 re¬ 
membered seeing our warning and 106 did not (see Table 5.6). Among the users who 
remembered seeing our warning, 45% were vulnerable, and among the users who did not 
remember seeing our warning, 56% were vulnerable. This difference is not statistically sig¬ 
nificant (p = .147 for Fisher’s exact test). We found no statistically significant difference 
within groups, either. 

Among the users who remembered seeing the warning. Table 5.7 summarizes the self- 
reported effects that the warnings had on those users. Of the 85 users who remembered our 
warnings, only 10 users’ responses (12%) indicated that our warnings helped them resist 
our attack. 38 of these users (45%) indicated that the warnings had little or no impact on 
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Group 

Warnings 

in email? 

User remen 

No 

ibered seeing 

Yes 

our warning? 

No response 

1 

N/A 

65.9% (27) 

31.7% (13) 

2.4% (1) 

2 

/ 

25.0% (10) 

62.5% (25) 

12.5% (5) 

3 


59.0% (23) 

41.0% (16) 

0.0% (0) 

4 

/ 

37.5% (15) 

57.5% (23) 

5.0% (2) 

5 


77.5% (31) 

20.0% (8) 

2.5% (1) 


Table 5.5: Number of users who reported remembering seeing our warnings. 



Safe 

Vulnerable 

Total 

Users who remembered seeing our warnings 

55.3% (47) 

44.7% (38) 

85 

Users who did not remember seeing our warnings 

44.3% (47) 

55.7% (59) 

106 


Table 5.6: Effect of warning recall on resisting our simulated attacks. Of the 200 users 
in our study, 9 users did not respond to this question. 


their decisions. 4 users (5%) indicated that the warning slightly influenced their decision 
making during the attack, but they ultimately followed the attack instructions. 7 users 
(8%) mentioned that the warnings made them “feel safer” at our site or be more careful in 
general. The responses of 11 users were inconclusive or did not clearly fit in one of these 
categories. 

5.3.4 User suspicion of our attacks 

To gauge users’ suspicion during our simulated attack, we asked users “During your 
interactions with UCB Movie Predictions, did you ever see something which looked suspi¬ 
cious or dangerous?” and “describe what your reaction was and if you did anything, what 
you did.” Overall, only 6 (15%) challenge questions users and 13 (8%) email users reported 
seeing anything suspicious during the study. Four of the challenge question users reported 
that the certificate warning caused their suspicion, but only 1 of those users resisted the 
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attack. 5 One challenge question user reported that the fact that the attack required her to 
re-register her machine made her suspicious. The 13 suspicious email users reported the 
attack instructions as the cause of their suspicion, but only 6 of those users resisted the 
attack. 

5.3.5 User reasoning during our attacks 

To help understand users’ thought processes during the simulated attack, we showed 
each user a screenshot of the attack instructions corresponding to her group and asked 
her: “If you followed the above instructions, explain why. If you chose not follow the 
instructions, explain why not. If you don’t remember this page or what you did, tell us 
what you don't remember.” We did not explicitly identify this as the “attack”. 

Challenge question users. Among the 38 vulnerable users in the challenge question 
group, 22 users (58%) said that they complied with the attack instructions because they 
thought it was what they needed to do to log in. Representative responses include: “Those 
were my challenge questions, so I answered them” and “I thought it was procedure to an¬ 
swer these questions.” 10 vulnerable users (26%) viewed the attack as an error that they 
should try to fix, e.g., “I thought the site’s cookie may have been erased which is why it 
wasn’t recognizing my computer, so I answered.” 4 vulnerable users (11%) trusted the 
Web site or indicated that since the site was associated with UC-Berkeley, it should be safe. 
Of the 3 challenge question users who resisted the attack, one user noticed the certificate 
warning and stopped, and the other two users did not respond to this question. 

Email users. Among the 66 vulnerable email users, 26 users (39%) complied with the 
attack instructions because they thought it was what they needed to do to log in. A repre¬ 
sentative response is “I copy and pasted the link because it said in bold to do so. It seemed 

5 Most browsers show certificate warnings in popup windows. Firefox 3 and Internet Explorer 7 present 
full screen interstitial warning pages, but Firefox 3 was not officially released until after we completed our 
study. Among the 41 challenge question users (who were the only users who saw certificate warnings - see 
Section 5.2.8), twenty five used Firefox 1 or 2, two used IE 6, six used IE 7, seven used Safari, and one used 
Epiphany. Among the 4 users who reported the certificate warning as the cause of their suspicion, three used 
Firefox 2 and one used IE 7. 
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like that was what I was supposed to do.” 11 vulnerable users (17%) viewed the attack as 
an error that they should fix, e.g., “I figured something was wrong with your registration 
system and thus followed instructions.” 20 vulnerable users (30%) trusted the Web site or 
indicated that since the site was associated with UC-Berkeley, it should be safe. 8 vulnera¬ 
ble users (12%) indicated that they complied with our attack instructions because they did 
not associate much risk with our Web site. 

Among the 93 email users who resisted the attack, the responses of 37 users (40%) 
indicated that although they may have recognized the instructions were different from pre¬ 
vious registrations, they decided to click the registration link first, despite instructions to 
the contrary, or did not read the attack instructions carefully. Representative responses in¬ 
clude: “I did not follow the instructions because it was easier to just click” and “Usually, 
in a verification email, you are supposed to click the link.” 

17 resisting users (18%) indicated that they did not notice the attack instructions. For 
example, “I never saw these instructions.” All except two of these users clicked on the 
registration link; the other two users timed out the attack. 

10 users (11%) cited our warnings as helping them resist the attack, e.g., “The Website 
and the email I received were telling me contradicting things so I just went with what the 
email told me” and “I didn't follow the instructions because they were contradictory to the 
warnings in the previous email.” 

We found evidence that 15 users (16%) initially considered the attack instructions, but 
eventually gave up because they found them too difficult, decided it was not worth the 
hassle, or made a mistake in following them. 5 users explicitly indicated this in their 
responses, e.g., “I did not because it was too much of a hassle” and “I figured I would see 
if the site would be on track later.” The remaining 10 users attempted to follow the attack 
instructions, but made a “mistake”, e.g., they forwarded our welcome email or copy and 
pasted a previously used registration link. Although we count these users as resisting our 
attack, they may be more likely to fall for future attacks than other users who resisted. 

The responses of 3 resisting users (3%) were hard to interpret. They stated that they 
followed the attack instructions, but we have no evidence that they attempted to do so; they 
all clicked on the link quickly. One possible explanation is that they did not actually notice 
the attack instructions, but attempted to please us during the survey or avoid appearing as 
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if they disregarded our instructions. 

5.3.6 Usability and security opinions of registration mechanisms 

To evaluate users’ impressions of the security benefits of the registration mechanisms, 
we asked users: “Rate how safe you would feel using a Web site which uses the following 
different login mechanisms: 1) Password for logins + no registration, and 2) Password for 
logins + <challenge questions/email> for registration”. The answer choices were: “Not 
secure at all”, “Somewhat secure”, “Fairly secure”, “Very secure”, and “I don’t know”. We 
summarize users’ responses in Table 5.8. 

To evaluate the usability of the registration mechanisms, we asked users: “Rate the 
convenience of each of the following login mechanisms: 1) Password for logins + no reg¬ 
istration, and 2) Password for logins + <challenge questions/email> for registration”. The 
answer choices were: “Nearly impossible or very annoying”, “Hard or slightly annoying”, 
“I could get used to it, “I hardly noticed it”, and “I don’t know”. We summarize users’ 
responses in Table 5.9. 

5.3.7 Ecological validity 

To evaluate the ecological validity of our study, we sought to determine how much 
risk users perceived while using our site. Since risk is subjective, we asked each user 
to tell us the biggest security concerns she has while browsing the World Wide Web and 
the precautions she takes to protect herself when logging into Web sites. Then, we asked 
her to rank how often and thoroughly she applies those precautions when logging into the 
following types of Web sites: banking, shopping, PayPal, Web email, social networking, 
and our study site. The answer choices were: “rarely”, “sometimes”, “usually”, “always”, 
and “I don't use this type of Web site”. We summarize the responses in Table 5.10. Users 
reported that they did not take the same level of precautions on our site as they do with 
other sites which handle money. Over 64% of users reported that they at least “usually” 
take security precautions at those sites, but only 27% of users said they at least “usually” 
took precautions at our study Web site. Users more closely associated the risks at our study 
Web site with a Web email site or social networking site. 
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In users’ responses to other questions, 2 users explicitly mentioned that they took pre¬ 
cautions because we had their PayPal email address, e.g., “I wanted to stay secure so that 
people couldn't come in and take my PayPal account.” However, 14 users explicitly men¬ 
tioned that they considered our study site to be low risk because they felt they had little 
to lose, e.g., “even if someone had hacked the site, what had I to lose? An experiment 
account? I was not particularly worried.” 

5.4 Analysis and discussion 

5.4.1 Ineffectiveness of our warnings 

Our results suggest that our warnings had little impact users' decisions, even when users 
had the opportunity to see warnings during the simulated attacks. We found no evidence 
that including warnings in registration emails helped users resist our attacks. Many users 
did not remember our warnings or indicated they had little impact on their decisions during 
the study. Although including contextual warnings in email seemed to improve the likeli¬ 
hood that a user would recall seeing them, we found no evidence that users who recalled 
seeing our warnings were more likely to resist our attack. Our results are consistent with 
a recent study by Egelman et al. which suggests that passive warnings such as ours are 
ineffective in helping users resist attacks [28]. 

Research from the warning sciences community suggests that if a warning does not 
sufficiently stimulate users, or if users cannot meaningfully process and apply a warning’s 
message, it will have limited effect [134]. Some responses from email users suggested that 
we failed to both get their attention and communicate a meaningful message. They assumed 
our warnings were similar to other “standard” warnings, or our warnings just made them 
feel our site was generally more secure. For example: 

• “I figured it was just standard stuff.” 

• “It looked like a standard confidentiality issue, so I didn’t think of it as anything 
particularly special.” 

• “I just chalked it up to general security advice and more or less forgot about it.” 
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• “It made me feel that the Website was more secure.” 

• “This bit of information made me feel like the site was trying to protect my privacy.” 

• “It did not affect my decisions much, but it did help the validity of the survey.” 

5.4.2 Lack of user suspicion 

Our attacks raised suspicion in only small percentage of users. Many of the other users 
had alternative interpretations. Some users saw the attack as the result of an error with 
the Web site, her browser, her computer, or the network, e.g., “I thought that the link was 
broken.” Some users did not view that complying with the attack instructions might be 
risky, but rather thought it was necessary for their own safety. For example, 

• “I followed the instructions because it was for my own safety.” 

• “I did because I wanted to stay secure so that people couldn’t come in and take my 
PayPal account.” 

• “The site is verifying I am who I say I am; I never thought of it in terms of me 
questioning the site’s identity.” 

Some users indicated they did not have a clear understanding of how the registration pro¬ 
cedure works and its purpose, and when they would be required to participate in the regis¬ 
tration ceremony. For example, 

• “I figured that because I switched connections, as I was using Berkeley’s wireless as 
opposed to my dormitory’s ethernet Internet, they needed to re-verify my account.” 

• “I followed the instructions because I assumed my password was wrong so the alter¬ 
nate method of login was by answering the security questions.” 

• “I figured it had been too many days since I'd signed in.” 

• “I answered them because I couldn’t remember if you guys said that we will ran¬ 
domly be asked to answer them in place of our password and login name.” 
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• “I remembered this page, and I followed the instructions because they are often used 
to verify a user if a username seems unsafe or has been tampered with.” 

• “The link in the email contains a data string that, when clicked, changes account 
details to confirm that that was a valid email address. Security benefits to the user 
may be minimal.” 

• “I think it prevents hackers from just creating accounts and using them but they would 
have to go to the extra step of doing the email registrations.” 

• “I actually didn’t think it had anything to do with the security of my money/identity.” 

These results are consistent with previous work which suggests that users have a limited 
understanding of Web security mechanisms, Internet social engineering attacks, and effec¬ 
tive defense strategies [24, 25, 49, 135]. This evidence supports our design principle for 
conditioned-safe ceremonies that argues designers should not assume users will be able 
to detect attacks, or sufficiently understand ceremonies to know when they should refrain 
from participating or perform voluntary defensive actions. 

5.4.3 The power of user conditioning and forcing functions 

Challenge question based registration conditions users to provide their answers when 
they are asked their challenge questions. The responses of 58% of the vulnerable challenge 
question users indicated that conditioning was the primary influence on their decision to 
comply with the simulated attack’s request for their answers. User responses of this type 
include: 

• “I answered the questions because I thought I was being asked to identify myself.” 

• “I answered it because it was required in order to log in.” 

• “I wanted to log in, so I answered the challenge questions.” 

This supports our hypothesis that challenge questions condition users to answer their chal¬ 
lenge questions whenever prompted. 
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In contrast, the responses of 56% of email users who resisted the simulated attack sug¬ 
gested that conditioning was a factor in their resistance. The responses of 40% of resisting 
email users suggested they may have noticed the attack was somewhat different from a 
normal registration, but either chose to ignore the attack instructions and click the link, or 
did not read the attack instructions carefully. User responses of this type include: 

• “I didn't follow the instructions because I didn't pay attention to this page (I just 
followed the usual procedures to register my computer).” 

• “I didn’t follow the directions because it sounded sketchy and I wanted to see what 
happened.” 

• “I must have glossed over the instructions to not click the registration email link, I 
didn’t think there would be two opposing instructions so I just went with the one that 
was more obvious.” 

• “I didn’t read it carefully, and instinctively clicked on the link in the email.” 

The responses of 16% of resisting email users suggested that they probably did not notice 
any differences between our simulated attack and a normal registration, and proceeded to 
click on the registration link in the email sent to them. User responses of this type include: 

• “I don’t remember ever seeing this page, but I think what I might have seen was 
simply that I thought this page was giving me the same instructions as the first time 
when I had to register my computer.” 

• “I don’t remember because it has been a hectic week. I just didn’t notice.” 

• “I don’t really remember this or I misread it.” 

• “It’s currently 2:20am and I just got back from 5 hours of dance practice. Honestly, 
I didn’t even see the instructions!!! How scary!” 

These results suggest that conditioning played a significant role in a large fraction of users’ 
decision making processes during our simulated attacks - to the benefit of email registra¬ 
tion, but to the detriment of challenge question based registration. 
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One factor our study did not control is to what degree challenge questions and click¬ 
ing on email links had conditioned users prior to participating in our study. Several sites 
currently implement challenge question based registration [9, 52, 123], and many use chal¬ 
lenge questions for password reset. Although we do not know of any sites that implement 
email registration for machine authentication, many Web sites send an email link to reset a 
user’s password or validate her email address [38]. We did not screen users based on previ¬ 
ous exposure to these mechanisms, but we did ask users whether they had previously used 
them. 80% of challenge question users and 70% of email users reported having used the 
respective mechanisms prior to participating in our study. However, we found no signif¬ 
icant correlation between previous exposure to these mechanisms and attack success rate. 
We leave better understanding of this issue as a subject of future work. 

5.4.4 Ecological validity 

We asked users to give feedback about their impressions of the study, and their re¬ 
sponses suggested that predicting popular movies can be fun and engaging. Some users 
expressed disappointment that we ended the study before they had the opportunity to make 
all 7 predictions. Some users admitted they had no idea as to the true purpose of the study, 
and no user claimed to have figured out that the study was security related. Based on this 
evidence, we argue the effects of demand characteristics were sharply diminished in our 
study. 

Our study created an experience of risk for some users, but many users indicated that 
the risk level they associated to our site was roughly equivalent to Web email or social 
networking sites, and below financial-related sites such as banking or shopping. Some 
users explicitly stated in their comments that they did not experience much risk during our 
study, e.g., “And even if someone had hacked the site, what had I to lose? An experiment 
account? I was not particularly worried.” Some users suggested that they felt safer at our 
site because it was associated with Berkeley, e.g., “I figured that since this was a Berkeley 
research affiliated Website, it would be safe.” Creating a significant experience of risk in 
studies like ours remains a challenge. 

Our design attempted to limit the effect of authority figures during the study by con- 
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ducting it in users’ “natural habitats”. One concern we had was that users would interpret 
the attack as instructions from us, the researchers. Although this is similar to the problem 
users face during a real phishing attack, academic researchers might appear more as an 
authority figure to a user than, say, a bank. There was evidence that this may have been 
an issue for some users, e.g., “I followed the instructions because I thought it was a legit¬ 
imate set of instructions from respected researchers who could not possibly have a motive 
to deceive me”, but maybe not for others, e.g., “My security is more important to me than 
their system problems.” We did not design our study to evaluate this issue in depth; further 
research is needed. 

5.4.5 Study limitations 

Our study had several limitations. Although we took great efforts to make our study as 
ecologically valid as possible (while remaining ethical), some users’ responses suggested 
we fell short in some aspects, most notably in simulating the experience of risk in the 
real world and completely eliminating the influence of authority figures. The size of the 
compensation may not have been large enough to warrant extra attention, and the fact 
that our Web site was implicitly associated with UC-Berkeley may have influenced users’ 
decisions. Also, since the vast majority of our users were undergraduates at UC-Berkeley, 
we cannot easily generalize our results to the general population. 

We acknowledge that there may be more effective attacks against email based registra¬ 
tion. One potentially effective attack might be to try to hijack users’ email accounts, but we 
did not implement this attack for ethical reasons. Another type of attack we did not evalu¬ 
ate is a prediction attack. In a prediction attack, the adversary creates the illusion that she 
can reliably predict the future. Being able to predict the future affords credibility, which an 
adversary may be able to exploit. If an adversary sends the user an email predicting that 
she will receive a registration email, but requests that she handles it unsafely, she may be 
more likely to comply. Stock market scams employing this technique are often effective. 

Although our results suggest that the notion of conditioned-safe ceremonies may be 
useful for helping users resist some types of Internet social engineering attacks, further 
research is necessary. We acknowledge that it remains to be seen whether the notion 
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of conditioned-safe ceremonies will be more generally applicable to other types of cere¬ 
monies, environments, and attack strategies. For example, it may be more challenging to 
develop conditioned-safe ceremonies to resist attacks whose only goal is to solicit and steal 
sensitive personal information. 

Our study collected a limited amount of information from each user. Since we never 
met our users, we could not directly observe users’ reactions, record comments, or probe 
for details during the study. Also, due to the remote nature of our study, there were many 
aspects we did not control that may have affected a user’s vulnerability to our attack. For 
example, we did not control how many times a user could log in or the number of computers 
she could register. These factors are difficult to control in a field study such as ours without 
introducing artificial constraints. 

Our study did not attempt to address any high-beam effects [31]. A high-beam effect 
refers to the situation where a driver may warn other drivers of upcoming law enforcement 
vehicles by flashing her high-beam headlights. A high-beam effect could arise in a user 
study that uses deception if users are debriefed at different times. A user who finishes 
earlier than others may inform them of the true purpose of the study before they have 
finished, which may introduce demand characteristics. 

We chose not to control for high-beam effects in our study because we anticipated 
several problems with waiting to debrief all the users at once. One problem is that it may 
have affected the quality of users’ exit survey responses. We expected that the duration 
for each user to finish the study would vary widely. This turned out to be the case. Many 
users finished within a week, but others took much longer, and one user never finished. 
If we had waited to debrief all the users at the same time, we were concerned the early 
finishers would have forgotten many details of the study - in particular, details about their 
behavior during the simulated attack. Another problem is that since our study employed 
deception, to ethically use a user’s data, we must debrief her on the study’s true purpose. 
If we had waited to debrief all the users at the same time, we were concerned that after 
early finishers had been paid, there may have been less incentive for them to respond to 
subsequent requests. 

For these reasons, we felt it was necessary to debrief each user immediately after the 
simulated attack, obtain reconsent, and give her the exit survey. Although we did not 
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attempt to measure any high-beam effects in our study, our potential subject pool was 
relatively large compared to our sample (200 vs. 1950), which at least minimizes the 
chances of any chatter among users. 
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Chapter 6 

Dynamic pharming attacks 


In this chapter, we show a new attack against Web authentication we call dynamic 
pharming. In a simple, static pharming attack, the adversary causes the victim’s requests 
for the target domain to connect to a server under its control, typically by arranging for the 
victim’s DNS queries for the target domain to return the adversary’s IP address. In contrast, 
in a dynamic pharming attack, the adversary causes the victim’s requests to connect to the 
legitimate server or its own, depending on the situation. 

We show how an adversary can use dynamic pharming to infect the victim’s browser 
with malicious Javascript and use this Javascript to hijack the victim’s session with the tar¬ 
get domain's legitimate server. Dynamic pharming enables an adversary to compromise 
all known authentication schemes for existing browsers, including passwords and machine 
authentication. In addition, the adversary can eavesdrop on sensitive content, forge transac¬ 
tions, sniff secondary passwords, and so on. This attack is particularly dangerous because 
it also affects cryptographic authentication mechanisms in browsers designed to resist man- 
in-the-middle attacks, such as client-side SSL. 

6.1 Overview 

Suppose the pharmer can control the results of DNS queries for www. vanguard. 
com, and users authenticate themselves to www. vanguard. com. The specific type of 
authentication used is not relevant to our attack. 
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Figure 6.1: An example of a dynamic pharming attack. (1) Initially, the pharmer ar¬ 
ranges for the victim’s DNS queries for www.vanguard.com to resolve to the pharmer’s IP 
address, 6.6.6.6. (2) Then, when the victim visits www.vanguard.com, the pharmer returns 
a trojan document containing malicious Javascript and a iframe referencing Vanguard’s 
home page. (3) The pharmer then updates the DNS entry for www.vanguard.com to the IP 
address of Vanguard’s legitimate server and denies subsequent connections from the vic¬ 
tim. (4) This causes the victim’s browser to renew its DNS entry for www.vanguard.com, 
and (5) load Vanguard’s legitimate home page in the iframe. (6) After the user authenti¬ 
cates herself, the malicious Javascript in the trojan document hijacks her session with the 
legitimate server. 


First, the pharmer initiali z es the DNS entry for www. vanguard. com to the pharmer’s 
IP address, say 6. 6 . 6 . 6 . Now suppose a user Alice visits https : //www. vanguard. 
com/index . html with the intention of authenticating herself. The user’s browser will 
attempt to establish an SSL connection, requiring the pharmer to present an X.509 cer¬ 
tificate. If the server certificate is not signed by one of the trusted CAs in the browser or 
the certificate’s CN does not match the server’s domain (i.e., www. vanguard. com), the 
browser will warn the user and ask her if it is safe to proceed. If the user heeds the warning 
and answers “no”, the browser will cancel the connection and the attack fails. If the user 
accepts the pharmer’s certificate—and there is substantial evidence that the user would (see 
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<html> 

<body> 

<script> 

-MALICIOUS JAVASCRIPT CODE- 


</script> 

<iframe 

src="https://www.vanguard.com/index.html"> 
</iframe> 


</body> 

</html> 


Figure 6.2: Dynamic pharming attack document. 

Section 2.1.3)—Alice’s browser will establish an SSL connection to the pharmer at 6.6.6.6 
and request index . html. 

In response, the pharmer returns a “trojan” index . html document. The purpose of 
this trojan document is to monitor and influence Alice’s subsequent interactions with the 
legitimate www. vanguard . com. We show the general structure of the trojan document 
in Figure 6.2. The attacker then causes the browser to load the legitimate https : // 
www. vanguard . com/index.html document into the <iframe> and display it to 
the user. 1 We discuss the details of how the attacker accomplishes this in the next section. 

Suppose the user now authenticates herself to www. vanguard . comin the <if rame> 
using any method supported in current browsers, e.g., password authentication or machine 
authentication via cookies or client-side SSL. 2 After authentication completes, the mali¬ 
cious Javascript in the outer document takes control and monitors the user’s interactions 
in the <if rame> with the legitimate server for www. vanguard . com. Since the outer 

Tframes are HTML elements which enable embedded documents. To prevent infinite recursion, most 
browsers disallow nesting where the URL of the framed document is the same as an ancestor. To address this 
issue, the attack could redirect the victim’s first request to https : //www. vanguard. com/ index2 . 
html or arrange so that the legitimate home page from www. vanguard. com loads in a separate window. 

2 We use www. vanguard. com only as an example here, and there is nothing specific about our attack 
to www. vanguard. com. 
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document and the <iframe> both have the same domain (www. vanguard. com) and 
same protocol (https), the SOP will allow the malicious Javascript running in the outer 
document to access the content in the <i f rame>. The trojan effectively hijacks control of 
Alice’s session - it can eavesdrop on sensitive content, forge transactions, sniff secondary 
passwords, etc. We show an example of a dynamic pharming attack in Figure 6.1. 

6.2 Attack details 

In this section, we present two ways in which the adversary can cause the user’s browser 
to load the legitimate https : / /www. vanguard. com/index . html document after 
previously connecting to the adversary. 

6.2.1 Method 1: Proxying 

One way to implement our attack is for the adversary to act as a port forwarding proxy 
tunnel between the user and the legitimate www. vanguard. com. After the adversary 
delivers the trojan attack document in Figure 6.2, it no longer responds directly to the 
user’s HTTP requests, but instead forwards SSL packets back and forth between the user’s 
computer and the legitimate www. vanguard. com. Since the first forwarded SSL packet 
will contain a session identifier unknown to the www. vanguard. com server, the server 
will re-initiate the SSL session with the user’s browser. The user’s browser will handle 
this automatically without any user involvement. After the SSL session renegotiation com¬ 
pletes, the user’s browser will be connected to the legitimate www. vanguard. com and 
the legitimate index . html will load in the if rame in the adversary’s trojan document 
(Figure 6.2). 

6.2.2 Method 2: DNS rebinding 

One drawback of the proxying attack in the previous section is that when the adversary 
starts to forward the user’s requests to www. vanguard. com, from Vanguard's perspec¬ 
tive, the requests appear to originate from the adversary’s IP address. This is undesirable 



74 


because it may increase the likelihood that Vanguard is able to detect the attack. For ex¬ 
ample, Vanguard may be suspicious if the adversary uses a blacklisted IP address, uses an 
IP address in foreign country, or proxies many attacks through a single IP address. In this 
section, we describe a technique exploiting DNS rebinding vulnerabilities that enables the 
adversary to cause all requests to Vanguard during the attack (both from the user and from 
the adversary) to appear to have originated from the user’s IP address. This enables the 
adversary to mount attacks (e.g., forge transactions) directly from the user’s IP address, 
significantly reducing the chance of detection. 

First, the pharmer initiali z es the DNS entry for www. vanguard. com to the pharmer’s 
IP address, say 6.6.6.6, and the pharmer also indicates in the DNS record that requesters 
should not cache this result, i.e., it sets the TTL=0. After the pharmer returns the trojan 
document to Alice, it updates the DNS entry for www. vanguard. com to the IP address 
of the legitimate server for www. vanguard. com, say 1.2.3.4. 

The adversary’s goal is to force the user’s browser refresh its DNS entry for www. 
vanguard. com, connect directly to Vanguard, and load the legitimate https : / /www. 
vanguard, com/index . html document into the <if rame>. However, one compli¬ 
cation to this method is Web browsers’ use of DNS pinning. With DNS pinning, a Web 
browser caches the result of a DNS query for a fixed period of time, regardless of the DNS 
entry’s specified lifetime. Browsers implement DNS pinning to defend against variants of 
the “Princeton attack” [42], also known as DNS rebinding attacks. In the “Princeton at¬ 
tack”, a malicious Web server first lures a victim who resides within a firewalled network 
containing privileged Web servers. 3 After the victim connects to the malicious server, the 
adversary changes its DNS entry to the IP address of a sensitive Web server located on the 
victim’s internal network. The SOP restricts malicious code from accessing other domains, 
but since the adversary’s domain now resolves to an internal IP address, this attack enables 
Javascript served by the adversary to access internal Web servers. 

DNS pinning poses a problem for dynamic pharming attacks because once a browser 
resolves a domain name using DNS, it will continue to use the IP address and ignore any 
subsequent changes the pharmer makes in the DNS system. However, since DNS pinning 

3 Assume these servers are accessible only to machines behind the firewall. 
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“breaks the Web” in certain scenarios, e.g., dual homed IPv6-IPv4 servers, dynamic DNS, 
and automatic failover, browsers implementers have recently relaxed their DNS pinning 
policies. 

Martin Johns discovered a technique for circumventing DNS pinning policies [61]. 
Johns discovered that a pharmer can force a victim to renew its DNS entry for a given 
domain on demand by rejecting connections from the victim, e.g., by sending an ICMP 
“host not reachable” message in response to subsequent attempts to connect to the server. 
The browser reacts by refreshing its DNS entry for the domain. 

In the basic dynamic pharming attack, we exploit Johns’s technique. After the pharmer 
delivers the trojan document to the user, it rejects subsequent requests from user’s machine 
and updates the DNS entry for www. vanguard. com to the IP address of the legitimate 
server. Now, when the user’s browser loads the <if rame>, it will first attempt to contact 
the pharmer, fail, refresh its DNS entry, receive the IP address of the legitimate server, and 
load the legitimate index . html document into the <if rame>. The attack continues as 
before. 

Using round robin DNS. To parallelize dynamic pharming attacks against multiple con¬ 
current users, it is inefficient to repeatedly update the DNS entry for www. vanguard. 
com. If the adversary has compromised a local, root, or authoritative DNS server, or 
changed the authoritative server of record for www. vanguard. com, the adversary can 
selectively respond with the pharmers IP or the legitimate server’s IP depending on the 
stage of attack. However, if the adversary only has the ability to change DNS entries for 
www. vanguard. com on a DNS server (e.g., by cache poisoning), this attack is unscal¬ 
able because the pharmer must update the DNS entry for each instance of the attack and 
reset it after the attack completes. 

A pharmer can use round robin DNS entries to make this attack scalable. A round robin 
DNS entry consists of multiple IP addresses for a single domain name. The DNS server 
returns an ordered list of the IP addresses in response to a query, but rotates the order for 
each response. Web sites typically use round robin DNS to implement load balancing or 
automatic failover. Browsers usually connect to the first IP address in the list, and this 
achieves some degree of load balancing among clients. When the connection fails, the 
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browser tries the next IP address on the list, until it successfully makes a connection. 

To leverage round robin DNS entries in a dynamic pharming attack, the pharmers cre¬ 
ates a round robin DNS entry containing two IP addresses: the pharmer’s IP and the le¬ 
gitimate server’s IP. Roughly half the DNS responses will be in the order: pharmer’s IP, 
server’s IP. In this case, the user will connect to the pharmer first, and the pharmer will 
deliver the trojan document. The pharmer rejects subsequent connections from the user, 
and the user’s browser will automatically fail over to the legitimate server, after which the 
attack proceeds as before. For the other half of responses, the user will be delivered di¬ 
rectly to the legitimate server and the pharming attack will silently fail. This shows how an 
attacker with the ability to replace a single DNS record, once, (e.g., by cache poisoning) 
can still attack thousands or millions of users. 

6.2.3 Discussion 

Dynamic pharming attacks do not leverage vulnerabilities in any particular authenti¬ 
cation mechanism; rather, they exploit how browsers currently enforce the SOP. Since 
dynamic pharming hijacks the victim’s session after she authenticates herself to the le¬ 
gitimate server, this attack most likely affects all known authentication mechanisms for 
current browsers, and probably all future ones as well. 

In some cases, pharming attacks can also steal users’ authentication credentials, e.g., 
passwords and machine authentication cookies. Since the users’ URL bar will show the 
correct domain name, even the most meticulous user might be fooled into revealing her 
password. Also, since browsers enforce the SOP based on domain names, pharmers can 
steal user’s machine authentication cookies for the target site. Although dynamic pharm¬ 
ing attacks against client-side SSL authentication do not enable pharmers to steal users’ 
authentication credentials (i.e., their private keys), as we have seen, they can compromise 
users’ sessions in real time. 

Some Web sites use Javascript to detect and prevent framing, e.g., 

if (parent.frames.length > 0) 

top.location.replace(document.location); 



77 


However, Javascript anti-framing techniques are not sufficient to resist dynamic pharming. 
Our attack does not depend on the use of iframes to be successful. For instance, the attacker 
could load the legitimate index . html in another tab or window. The SOP still allows 
malicious Javascript access to the second window, and this situation is much harder for the 
legitimate site to detect. 

6.2.4 Proof of concept implementation 

We implemented a proof of concept dynamic pharming attack using a pair of Apache 
SSL Web servers (i.e, a pharmer and a target) and round robin DNS. We tested the at¬ 
tack against two browsers: Firefox 2.0 running on Debian GNU/Linux 3.1 and Microsoft 
Internet Explorer 7.0 running on Windows XP Professional SP2. After the adversary de¬ 
livers the trojan document, she refuses further connections from the client. This causes the 
browser to renew its DNS entry for the target domain and connect to the legitimate server, 
after which the adversary hijacks the session with the malicious Javascript in the trojan 
document. We found both browsers to be vulnerable to this dynamic pharming attack. 

6.3 Library import and data export pharming vulnerabil¬ 
ities 

Pharmers can also launch attacks by exploiting browsers’ library import and data export 
features. Library import allows Web pages to import additional documents (e.g., Javascript 
libraries and cascading style sheets), for example: 

<script type="text/javascript" 

src="https://xyz.com/login.js"> 

Data export allows Web documents to export information back to servers, e.g., by sub¬ 
mitting a form. Many library import and data export features are not governed by the 
same-origin policy. 4 Browsers allow documents to import libraries from and export data to 

4 An example of an import/export mechanism that is governed by the same-origin policy is 
XMLHTTPRequest, which is restricted to sending and receiving requests to the domain of the enclosing 
document. 
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any domain. For example, a document from abc . com could import Javascript both from 
its own domain and from xyz.com and submit a form to f oo . com. 

Jackson and Barth discuss how pharmers can exploit these features [53]. One problem 
is that if a document imports a library containing executable code, e.g., Javascript, the code 
inherits the protection domain of the enclosing document. For example, if a pharmers hi¬ 
jacks control of DNS for xy z . com and a document from abc . com imports libraries from 
xy z . com, the pharmer can inject malicious Javascript into the document from abc . com, 
giving it access to other documents and objects from abc . com. A Web document can be 
vulnerable to these pharming attacks even if it only imports libraries from its own domain 
(either explicitly or via relative URLs). For example, suppose a pharmer has hijacked DNS 
for abc . com. If a document from abc . com imports a library from abc . com, instead 
of interposing on the main document, the pharmer can wait until the browser requests the 
library and supply a hie containing malicious Javascript. 

Pharmers can exploit similar vulnerabilities in data export features, such as form sub¬ 
mission. If pharmer hijacks control of DNS for the domain of a form’s target, it intercept 
the form submission and steal any sensitive information contained within, including cook¬ 


ies. 
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Chapter 7 

The locked same-origin policies 

Since dynamic pharming hijacks a user’s session after initial authentication completes, 
this attack is independent of the authentication mechanism and affects all known authen¬ 
tication schemes for current browsers, including passwords, authentication cookies, and 
client-side SSL. It is therefore unlikely that any future Web authentication protocol devel¬ 
oped for existing browsers will resist dynamic pharming either. Although dynamic pharm¬ 
ing attacks leverage the implementation details of DNS pinning, “fixing” DNS pinning 
to resist DNS rebinding attacks is challenging. DNS pinning has a lengthy and contro¬ 
versial history in Firefox and Mozilla [83], and the current implementation is an explicit 
compromise to support dynamic DNS and round robin DNS for failover [82, 84]. From 
the browser’s point of view, a dynamic pharming attack is indistinguishable from a fail¬ 
ure of a site and DNS round robin recovery. Lastly, it is unlikely Web sites can resist 
dynamic pharming attacks effectively. The adversary has the advantage of loading her doc¬ 
ument first; she can read and modify all of the legitimate server’s documents in the victim’s 
browser, as well as control their execution environment. 

To resist dynamic pharming, we must address the root of the problem: we must upgrade 
browsers’ same-origin policy (SOP). A SOP based on domain names will fail because 
pharmers control the mapping from domain name to subject. For Web objects retrieved 
over insecure HTTP, it is unclear how the browser can distinguish a pharmer from the 
legitimate server. However, for objects retrieved over SSL, which we refer to as locked 
Web objects, we argue browsers should enforce the SOP using cryptographic identity. 



80 


A YURL based same-origin policy. A YURL [18] consists of a URL and a public key 
hash. The intention is for browsers to use the public key hash to authenticate the Web server 
using SSL before requesting the URL. For example, if browser requests a YURL for xyz . 
com, the browser uses DNS to resolve xyz . com to an IP address, and then establishes an 
SSL connection with the server. After a connection is established, the browser compares the 
public key hash in the YURL against the public key of the server. If the hash is consistent 
with the server’s public key, the browser proceeds with the request; otherwise it cancels the 
connection with no opportunity for user override. 

Upgrading browsers’ same-origin policy to support YURLs is sufficient to resist dy¬ 
namic pharming attacks against locked Web objects. The basic idea is to extend the notion 
of origin to include the public keys referenced in YURLs. A YURL based SOP would not 
consider two objects from the same origin unless: 1) their domains match, and 2) both ob¬ 
jects were retrieved over SSL from servers with the same public key. This restriction would 
only apply to locked Web objects; for non-SSL Web objects, the legacy SOP (namely, using 
domain names) would still apply. 

Unfortunately, this solution imposes a non-trivial deployment burden on Web sites. To 
take advantage of a YURL based SOP, a Web site must alter its servers to use YURLs in 
all places it currently uses HTTPS URLs, including all instances of library import and data 
export; otherwise the site may be vulnerable to the attacks discussed in Section 6.3. Since 
users do not type YURLs into the URL bar, a Web site must be careful to redirect all non- 
YURLs to YURLs before performing any sensitive operations, e.g., setting authentication 
cookies. These alterations must also be backwards compatible with legacy browsers. 

Our goal is develop extensions to browsers’ same-origin policy that provide comparable 
security to a YURL based SOP, but minimize the deployment burden for Web sites. Ideally, 
our goal is to develop solutions that require browser changes, but no or minimal server 
changes. In addition, we require our solutions to be backwards compatible with existing 
browsers and servers. 

Towards achieving this goal, we propose two locked same-origin policies , which like 
the YURL based SOP, enforce access control based on cryptographic identity. We first 
present the weak locked same-origin policy, which isolates a domain’s locked Web objects 
with valid certificate chains from objects with invalid chains. We then present the strong 
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locked same-origin policy, which enforces access control based on Web sites’ SSL public 
keys. 

Both policies only apply new restrictions to locked Web objects. For non-SSL Web 
objects, the legacy SOP (namely, using domain names) still applies. Like the legacy SOP, 
both locked SOPs deny unlocked Web objects (that is, objects not retrieved over SSL) 
access to locked Web objects. We summarize our policies in comparison to the legacy SOP 
in Table 7.1. 


7.1 The weak locked same-origin policy 

The legacy SOP currently allows access to locked Web objects only from other locked 
Web objects originating from the same domain. 1 However, the legacy SOP does not distin¬ 
guish between locked Web objects retrieved from a legitimate server with valid certificate 
and those from a pharmer spoofing the server’s domain name with an invalid certificate, 
and will allow access if the user ignores any certificate warnings. To resist pharming at¬ 
tacks, the weak locked SOP augments the legacy SOP by tagging each locked Web object 
with a validity bit indicating whether the certificate chain corresponding to the SSL connec¬ 
tion over which the object was retrieved contained any errors (e.g., self-signed certificate, 
CN/domain mismatch), irrespective of how the user responded to any certificate warnings. 
Then, the browser allows a locked Web object to access another locked Web object if and 
only if 1) the legacy SOP would allow access and 2) the validity bits match. 

7.2 The strong locked same-origin policy 

With the strong locked SOP, we propose browsers augment the legacy SOP by tagging 
each locked Web object with the public key of the other endpoint of the SSL connection 
(i.e., the Web server). Then, the browser allows a locked Web object to access another 
locked Web object if and only if 1) the legacy SOP would allow access and 2) the associ- 

1 Exception: if a Web site sets a non SSL-only cookie (i.e., without the secure attribute) over an SSL 
connection, then this policy allows the same domain to access the cookie over non-SSL connections as well. 
Essentially, a non SSL-only cookie set over an SSL connection gets downgraded to an unlocked Web object. 
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ated public keys match. The strong locked SOP was inspired by Key Continuity Manage¬ 
ment [39, 45, 102, 143], a technique for associating public keys with subjects and taking 
defensive action when a subject’s public key unexpectedly changes in a future interaction. 

The strong locked SOP is similar to the YURL based SOP, but unlike the YURL based 
SOP, the strong locked SOP does not require Web sites to upgrade all instances of URLs 
into YURLs. The tradeoff is that while a YURL completely specifies a Web object’s origin, 
an object’s origin under the strong locked SOP is not completely specified until after the 
browser connects to the server and determines its public key. We discuss the consequences 
of this tradeoff in Section 7.7. 


7.3 Security analysis 

7.3.1 Weak locked same-origin policy 

If a Web server hosting domain D (i.e., the target domain) uses a valid X.509 certificate 
signed by a trusted root CA, the weak locked SOP resists phishing, pharming, and active 
attacks against D's locked Web objects (i.e., illegitimate access by the adversary’s Web 
objects) as long as the adversary is unable to obtain a valid certificate for D. The weak 
locked SOP resists phishing attacks because a phisher has a different domain name. For 
pharming and active attacks, the adversary can arrange for her Web objects to have the 
same name as the target domain, but if she does not have a valid certificate for the target 
domain, the validity bit will be false, while the validity bit of the Web server’s locked 
objects will be true. Thus, the adversary is denied access. 

If the legitimate target site uses an invalid X.509 certificate (e.g., expired, CN/domain 
mismatch, or self-signed), the weak locked SOP provides no additional protection over 
the legacy SOP. It resists phishing attacks, but does not protect against pharmers or active 
attackers. 

In contrast to the legacy SOP, the weak locked SOP does not depend on users cor¬ 
rectly answering prompts in response to certificate errors (e.g., if an adversary presents a 
self-signed certificate with a spoofed domain name). The browser tags locked Web objects 
according to the validity of the server’s certificate and its domain name, and nothing else. 
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However, the weak locked SOP does assume that the trusted root CAs do not issue valid 
certificates for D to unauthorized parties. Although CAs take measures to prevent this, 
mistakes have been made in the past [80]. 


7.3.2 Strong locked same-origin policy 

If a Web server hosting domain D uses an X.509 certificate with public key PK, the 
strong locked SOP resists phishing, pharming, and active attacks against D 's locked Web 
objects as long as the adversary does not know the corresponding private key for PK. As 
with the weak locked SOP, the strong locked SOP resists phishing attacks because a phisher 
has a different domain name. In order to access D’s locked Web objects, the adversary 
must pharm D and also arrange for its own objects to be tagged with PK. However, the 
browser will only do this if 1) the adversary presents a X.509 certificate with PK, and 2) 
the browser and adversary can successfully establish an SSL connection. If the adversary 
tries to present a certificate for PK and she does not know the private key corresponding 
to PK, she will not be able to successfully complete the SSL handshake; the browser 
will automatically cancel the connection with no option of user override. Thus, since the 
browser will only tag the adversary’s locked Web objects with a public key different from 
PK, the browser will deny the adversary access to D’s locked Web objects. For the same 
reason, the strong locked SOP protects D’s locked Web objects against active attackers as 
well. 

As with the weak locked SOP, the strong locked SOP does not depend on users correctly 
answering prompts in response to certificate errors. Furthermore, in contrast to the weak 
locked SOP, the strong locked SOP does not require a Web site to trust root CAs not to 
issue certificates to unauthorized parties for its domain. Enforcement relies only on servers’ 
public keys. 
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7.4 Deployability analysis 

If our locked same-origin policies are to be successful, they need to be easy to deploy 
and backwards compatible; they should not “break the Web” because of problems with 
deployment or interoperability with existing Web servers. Since no browser developer 
is likely to embrace a policy that makes her browser incompatible with existing Web sites, 
legacy Web servers had better continue to work even when visited with locked SOP enabled 
browsers. 

Our policies are more restrictive than the legacy SOP, but we only want to deny access 
to an attacker - never the legitimate server. We will “break” a Web site if there is a situation 
where our policy would deny a legitimate server access to one of its locked Web objects, 
but the legacy SOP would allow access. 

There are a few situations where our policies could potentially break a Web site. For ex¬ 
ample, suppose server A for xyz.com has an valid certificate, but server B for xyz.com 
has an invalid certificate (or vice versa). Then the weak locked SOP would deny Javascript 
from server A from accessing an HTML document from server B, but the legacy SOP 
would allow access. This situation might arise if xy z . com uses round robin DNS for load 
balancing and a browser request objects from both servers during a session. Note that the 
weak locked SOP will not break a domain which uses invalid certificates on all its servers 
(e.g., it uses self-signed certificates) since objects from these servers have equivalent valid¬ 
ity: they are all invalid. 

If server A and server B use different public keys, then the strong locked SOP would 
also deny access. However, the strong locked SOP does not necessarily require the domain 
to use certificates issued by a root CA trusted by browsers. As long as all servers use the 
same public key, the Web site can use certificates issued by a root CA untrusted by browsers 
or self-signed certificates. 

7.4.1 An SSL server survey 

To evaluate the deployability of our policies, we must determine how many sites we 
could potentially break; in other words, how often the above configurations actually arise 
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in practice. To measure this, we surveyed SSL servers in the real world to determine how 
many servers may not currently interoperate with our policies. We constructed a sample 
of SSL servers by first crawling the Web, starting from a list of major news, portal, and 
financial sites. Whenever we found an HTTPS link, we added the domain in the link to 
our sample. For the sake of simplicity, we restricted our study to the following top-level 
domains: com, org, net, gov, edu, biz, info, and name. We excluded international 
top-level domains. We found 14651 fully qualified SSL domains from 6192 second-level 
domains. 2 This corresponds to roughly 6.5% of the number of SSL domains found by 
the more extensive monthly SSL survey conducted by E-Soft and securityspace. 
com [109], 

We are primarily interested in finding domains hosted by multiple servers, since it is 
these domains that our policies could potentially break. We can discover some servers by 
looking for use of round robin DNS; if a server uses round robin DNS, a DNS query returns 
a list of IP address. However, Akamai-style load balancing often considers the physical 
location of the DNS querier and may only return the IP address of most appropriate server. 
To take Akamai-style load balancing into account, we constructed a list of servers for each 
domain by requesting recursive DNS queries to 15 geographically distributed public DNS 
servers [125]. 3 Of the 14651 domain names, we found 1464 that resolved to multiple IP 
addresses. For each of these domains, we established an SSL connection to each of the 
domain’s servers and recorded each server’s certificate chain and public key. 

7.4.2 Certificate chain validation: Firefox and IE 

The next step was to validate the certificates we collected. To maximize the practical 
relevance of our study, we simulated the validation procedures of Firefox 2.0 and Inter¬ 
net Explorer 7.0. The validation procedures of Firefox and IE are close to the process we 
described in Section 2.1.3, but there are some differences in how each browser handles 
missing and expired intermediate CA certificates. Intermediate CA certificates are certifi- 

2 For our study, a second-level domain means the last two components of a non-international fully qualified 
domain name, e.g., yahoo . com. 

5 A limitation of this approach is that we cannot discover multiple servers behind a front-end load balancer 
with a single IP address. 
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Browser 

Session caching 

of CA certs 

Persistent caching 

of CA certs 

Uses AIA 

IE 

/ 

/ 

/ 

Firefox 

/ 




Table 7.2: Summary of browser mechanisms used to address missing and expired 
intermediate CA certificates. AIA refers to the optional Authority Information Access 
X.509 extension. 

cates issued by a CA’s root certificate which it uses to directly issue certificates to Web 
servers. This results in certificate chains of length 3 or more. Since most browsers only 
ship with root CA certificates, to guarantee a client can verify its chain, a server must also 
send any intermediate CA certificates in addition to its own. 

Unfortunately, many servers are not configured to send intermediate CA certificates. 
Also, there are several widely used intermediate CA certificates which have expired, and 
although the CA has reissued a replacement with the same name (and often, the same 
public key), many servers have not updated them and are still sending the expired version. 
We found that Firefox and Internet Explorer handle these situations slightly differently. We 
determined each browser’s validation procedure through source code analysis, empirical 
testing, and various public sources [40, 85, 86]. 

First, both Firefox and Internet Explorer cache the intermediate CA certificates they 
encounter during a user’s browsing session and use this cache to help verify certificate 
chains. This means if the user visits a site with a missing intermediate CA certificate, 
and previously in the session, the user visited a different site using the same intermediate 
certificate, the browser uses the cached copy to verify the chain. In addition, if the user 
visits a site which sends an expired intermediate CA certificate, both Firefox and Internet 
Explorer will automatically replace it with the more recent version if they have seen it 
previously in the session. 

Internet Explorer takes some additional measures to address missing intermediate CA 
certificates that Firefox does not. First, in addition to caching intermediate CA certificates 
within a session, Internet Explorer caches these certificates persistently, across sessions. 
Second, Internet Explorer takes advantage of the Authority Information Access (AIA) ex- 
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tension included in some X.509 certificates. The AIA extension “indicates how to access 
CA information for the issuer of the certificate in which the extension appears” [106]. We 
found for many server certificates issued by an intermediate CA certificate, they include the 
AIA extension with a URL for the intermediate CA certificate, and Internet Explorer auto¬ 
matically downloads and uses it to verify the chain. Firefox does not use the AIA extension. 
It is unclear exactly why not, but discussions on Mozilla Bugzilla suggest it might be be¬ 
cause some of the Mozilla developers believe the AIA standard is not well specified [85]. 
As a result of these additional mechanisms in Internet Explorer, Firefox generates more cer¬ 
tificate warnings on average for sites with missing or expired intermediate CA certificates. 
We summarize these differences in Table 7.2. 

7.4.3 Evaluation results 

Weak-locked same-origin policy 

To evaluate the deployability of the weak locked SOP, we validated the servers’ cer¬ 
tificate chains in our survey using two procedures: Pessimistic Validation and Optimistic 
Validation. Pessimistic Validation models the worse case scenario: a Firefox user visits 
a Web site with a missing or expired intermediate CA certificate at the start of a session, 
or a user freshly installs IE and visits the same site, and the server certificate does not 
support AIA. Through empirical analysis we identified 18 widely used intermediate CA 
certificates, and for Optimistic Validation, we assume the user’s browser has cached valid 
versions of these certificates. We intend Optimistic Validation to model a long Firefox 
session or a “broken in” Internet Explorer installation with a substantial intermediate CA 
certificate cache. 

Then, for the 1464 fully qualified SSL domains which use multiple servers, we counted 
the number of domains which had servers with both valid and invalid certificate chains, 
since it is these domains that the weak locked SOP may break. Using Pessimistic Valida¬ 
tion, we found 8 such domains, and for Optimistic Validation we found 4 domains. For 
each of the other 1456 domains, its servers either had all valid certificates or all invalid 
certificates. 

The difference between the Optimistic and Pessimistic Validation results means we 
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Policy 

Percentage of potentially 

non-interoperating domains 

Weak locked SOP 

0.05% 

Strong locked SOP 

0.6% 


Table 7.3: Summary of our deployability analysis of the locked same-origin policies 
using a sample of 14651 SSL domains. This table shows the percentage of domains in 
our survey which may not correctly interoperate with the locked same-origin policies. 

found 4 domains that contained a mix of servers with missing or expired intermediate 
certificates and correctly configured servers. Of the 4 remaining domains which still cause 
problems with Optimistic Validation, 3 are probably the result of virtual hosting issues. 
For example, of the 3 servers we found for signin . half . ebay . com, one had a valid 
certificate, and the other two had CN/domain name mismatch problems. These two servers 
presented certificates for signin . ebay. com. The remaining problem domain was the 
result of an expired certificate on one of its servers. When the domain’s administrators 
updated their certificates, they probably overlooked this server. 

This means the weak locked SOP would potentially break at most 0.05% of the SSL 
domains we found in our survey. These results are strong evidence that browsers could 
enforce the weak locked SOP today and still interoperate with the vast majority of Web 
sites while providing increased protection against pharming attacks. Furthermore, since 
the number of problem domains is relatively small, browser developers can conceivably 
work with these domains’ administrators to make their servers consistent. In conclusion, 
we can safely deploy the weak locked SOP in a way which requires minor browser changes, 
but does not require changes to the HTTP specification, SSL, or Web servers. 


Strong locked same-origin policy 

To evaluate the deployability of the strong locked SOP, we counted the number of fully 
qualified SSL domains with multiple servers that do not use the same public key on all 
of the servers. We found 83 such domains, representing 0.6% of the total number of SSL 
domains in our survey. This is problematic for two reasons. First, it represents an order 
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of magnitude more servers than that are affected by the weak locked SOP. Second, unlike 
before, these servers are not necessarily misconhgured, so browser developers cannot work 
with the domain's administrators to “fix” the problem. Using a different key on each server 
is good security practice, since it limits the scope of key compromise. In fact, VeriSign 
explicitly recommends customers use different public keys on each server [124], 

Another problem concerns certificate expiration. The business model of many CAs is to 
issue certificates that are valid only for a relatively modest period of time, e.g., one or two 
years, and require customers to renew their certificates when they expire. When Web sites 
renew their certificates they often follow good security practice and generate a new public 
key. Since the strong locked SOP applies to all locked Web objects, if a Web site uses 
persistent SSL-only cookies to authenticate users (see Section 7.10), every user’s cookie 
will simultaneously “expire” (i.e., become inaccessible by the server) when the site starts 
using the new public key, regardless of the value of the cookie’s expires attribute. 

Based on this evidence, we conclude browsers cannot currently enforce the strong 
locked SOP without potentially breaking a significant number of Web sites. However, this 
does not mean that deploying the strong locked SOP is hopeless; it only means a browser 
must first get the site’s explicit cooperation and approval to enforce it. In the next section, 
we describe a simple incrementally deployable solution using policy hies for the strong 
locked SOP which supports multiple public keys and key updates. 

7.5 Policy files for supporting multiple keys and key up¬ 
dates 

We propose an incrementally deployable solution where a Web site can opt in to the 
strong locked SOP; then, browsers which support the policy can safely enforce it without 
the risk of unintentionally breaking the site. To opt in, we propose a site’s servers post a 
policy hie with a well-known hie name, say pk. txt. Sites can notify clients that they 
support the strong locked SOP, e.g., via a special HTTP header. Legacy clients will ignore 
this header. Compatible Web clients can retrieve the pk . txt hie over SSL, parse it, and 
start enforcing the strong locked SOP for that domain. 
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If all the domain’s servers use the same public key and persistent objects are not an 
issue, a site can simply post an empty pk.txt, since it will already interoperate with 
the strong locked SOP. If the site uses multiple servers, labeled i = 1... n, with different 
public keys, then pk . txt on server k contains a list of the form: 

(pki, {pk k } skl ), (pk 2 ,{pk k } sk2 ), ..., (pk n , {pk k } skn ) 

where pk k is the public key of the server hosting this pk . txt file and {pk k } ski represents 
a signature of pk k by the secret key corresponding to pk,. The browser then verifies each of 
the signatures, and for i = 1... n, if the i th signature is valid, then it considers pk k to “speak 
for” pki. We then extend the strong locked SOP with the following rule: a browser allows 
a locked Web object tagged with (D, pkj ) to access another locked Web object tagged with 
(I), pki ) if a policy file attests that pkj speaks for pki. 

Note that pk.txt cannot simply list the public keys ( pk\,pk 2 ,... ,pk n ); otherwise 
a pharmer can serve the same file to a victim, and the victim’s browser will infer that 
the pharmer’s public key speaks for each of the keys of the legitimate servers. However, 
since the pharmer does not know the legitimate servers’ private keys, it will not be able 
to generate any valid signatures required for the victim’s browser to infer the “speaks for” 
relation. 

Policy files also address the problem of public key updates discussed in Section 7.4.3. 
For example, suppose a Web site wants to renew its certificate with a new public key pk new . 
Then, several months before the certificate expires, the site can include (pki, {pk new } ski ) in 
its pk.txt files for each server i, i = 1... n. Then, users that retrieve pk.txt during 
this transition period will not “lose” persistent objects tagged with an old public key. 

In conclusion, policy files address the deployability problems with the strong locked 
SOP we identified in Section 7.4.3. They enable us to enforce the strong locked SOP 
in current browsers in a way which is incrementally deployable and backwards compati¬ 
ble with legacy servers. The strong locked SOP in conjunction with policy files requires 
browser changes and server configuration changes for sites wishing to take advantage of 
the policy, but does not require changes to the HTTP specification or SSL. 
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7.6 Support for subdomain object sharing 

Up until now we have implicitly assumed a Web site consists of a single fully qualified 
domain name, e.g., www. xy z . com. More generally, a Web site might be composed of 
several domain names, e.g., mail. xyz . com, www. xyz . com, login . xyz . com, and 
the legacy SOP supports some exceptions which enable these sites to share information 
among subdomains through certain Web objects. For example, a user might authenticate 
herself to the server for login . xyz . com, and this server will set a domain cookie with 
domain=xy z . com for authenticating the user to the other subdomain servers. The user’s 
browser will allow any subdomain of xyz . com to access this cookie. Another way sub- 
domains can share information is by setting the DOM property document. domain. For 
example, if a document from www. xyz . com sets document. domain=xyz . com, the 
browser permits any object from a subdomain of xyz . com to access the document. 

However, these domain sharing mechanisms are vulnerable to pharming attacks. For 
example, if an adversary pharms any host name in xyz . com, she can steal users’ domain 
cookies for xyz . com. Ideally, we would like to enforce our locked same-origin policies 
in these situations as well. Fortunately, extending the strong locked SOP to support subdo¬ 
main sharing is straightforward with policy files. The site simply adds the servers’ public 
keys to its policy files and we extend the strong locked SOP with the following rule: if S is 
locked Web object hosted by server l and is designated to be shared among subdomains of 
a higher-level domain TD (e.g., xyz . com), a browser allows a locked Web object tagged 
with (D, pkj) to access S if D is a subdomain of TD and a policy file attests that pkj speaks 
for pk t . 

Unfortunately, it is not clear how to extend the weak locked SOP to support shared 
domain objects without any server cooperation. An natural candidate extension would 
be to allow access if both subdomain servers have valid certificates or invalid certificates. 
However, we must have confidence this policy will not “break the Web” and not deny access 
to a legitimate server when the legacy SOP would allow access. Roughly, this would require 
for each higher-level domain, either all its subdomain servers have valid certificates or all 
its subdomain servers have invalid certificates. Unfortunately, our survey survey shows this 
is far from the case. Of the 6192 second-level SSL domains we found, over 1000 did not 
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satisfy this property. This means for browsers enforcing the weak locked SOP, they must 
default back to the legacy SOP for shared domain objects, which provides no protection 
against pharming. 

7.7 Support for library import and data export 

In Section 6.3, we discussed pharming vulnerabilities with library import and data ex¬ 
port features in browsers. Many import and export features are not governed by the same- 
origin policy. Web sites can import libraries from and export data to any domain. Since 
Web sites routinely use import and export features, without additional protection mecha¬ 
nisms, the locked same-origin policies alone are insufficient to protect sites from pharming 
attacks. Using YURLs for all library import and data export operations resists pharming 
and active attacks because a YURL explicitly specifies the public key of the server host¬ 
ing the library or receiving the data. However, as we discussed at the beginning of this 
chapter, YURLs may be troublesome for Web sites to manage and deploy. In this section, 
we discuss some alternative solutions for protecting library import and data export features 
that work in conjunction with the locked same-origin policies to resist pharming and active 
attacks. 

7.7.1 With the weak locked same-origin policy 

The weak locked same-origin policy isolates a domain’s locked Web objects with valid 
certificate chains from objects with invalid chains. Our goal is to provide similar guaran¬ 
tees for Web documents that use library import and data export features. Currently, many 
browsers will display a warning if a document imports from or exports to a server with an 
invalid certificate. This warning may be a “ribbon-type” warning, a pop up warning, or a 
full page warning, depending on the browser and the circumstances. Many of these warn¬ 
ings offer a user override option, which allows the import or export to continue, potentially 
compromising the integrity of the Web application during a pharming or active attack. 

One solution to this problem we propose is a variant of YURLs we call weak YURLs. 
A weak YURL is an HTTPS URL together with a bit flag indicating that the browser should 
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only connect to a server that provides a valid certificate for the domain in the URL. The 
security properties of weak YURLs are similar to those the weak locked same-origin policy. 
In conjunction with the weak locked same-origin policy, weak YURLs resist pharming and 
active attacks against library import and data export operations as long as an adversary can 
not obtain a valid certificate for the target domain. If the adversary attempts to intercept the 
import or export operation, it will cause a certificate error, and the browser will cause the 
operation to fail without any option for user override. 

An advantage of weak YURLs over “strong” YURLs is that weak YURLs are easier to 
manage and deploy. Strong YURLs require Web sites to maintain an accurate list of public 
keys for the servers involved in library import and data export operations. Some Web sites 
import libraries from “third parties”, for example the home page for www. paypal. com 
includes the following import: 

<script type="text/javascript" 

src="https://www.paypalobjects.com/..."> 

If the server for www. paypalob jects . com updates its public key and www. paypal. 
com fails to update its “strong” YURL to use the new public key, this import will fail, 
potentially breaking the functionality of the application. Weak YURLs avoid this problem. 
As long www. paypalob jects . com always uses a valid certificate, then a weak YURL 
will always resolve successfully (except during an attack, of course). One easy protocol 
to take advantage of weak YURLs is for a Web site to include a special HTTP or HTML 
header indicating that compliant browsers should interpret all URLs in the document as 
weak YURLs. 

A disadvantage of this solution is that unlike the weak locked same-origin policy, weak 
YURLs require minor server modifications. However, if the enclosing document was 
fetched from a server with a valid certificate, we argue that browsers could by default safely 
interpret all “first-party” URLs (i.e., URLs with the same domain as the enclosing docu¬ 
ment, including relative URLs) in import and export operations as weak (valid) YURLs 
with little risk of breaking those applications. In other words, if the enclosing document 
was fetched from a server with a valid certificate, all import and export operations (in the 
enclosing document) to the same domain must also be to a server with a valid certificate, 
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otherwise the operation will fail with no chance of override. This would only cause a prob¬ 
lem if a domain has multiple servers with a mix of valid and invalid servers, and only 0.05% 
of the domains in our survey had this property (Section 7.4.3). This policy does not require 
any server changes and helps Web applications that only import from and export to their 
originating domain resist pharming and active attacks. 

7.7.2 With the strong locked same-origin policy 

To resist pharming and active attacks against import and export operations in conjunc¬ 
tion with the strong locked same-origin policy, we propose extending the policy file mech¬ 
anism in Section 7.5. To authorize a domain for import or export operations, a Web site 
includes the domain and its associated public keys in the policy file. Although, conceptu¬ 
ally this is similar to using YURLs for all import and export operations, we argue that this 
solution is easier for sites to deploy than YURLs. Policy hies allow sites to consolidate the 
public keys of the import/export servers in a single location without requiring significant 
changes to their server infrastructure or Web application framework. 

7.8 Caching 

Browsers do not by default persistently cache HTTPS Web objects (except SSL-only 
cookies), but they do use session caching. As we demonstrated in Chapter 6, adversaries 
can use the cache to launch subtle dynamic pharming attacks. To resist attacks involving the 
browser cache, compliant browser must be careful to apply the locked same-origin policies 
to cache elements as well. 


7.9 Limitations 

Our locked same-origin policies do not address attacks where the adversary tricks a 
victim into installing malicious software such as executable malware, an ActiveX plugin, 
or a browser extension. These objects usually execute with elevated privileges and are not 
governed by the SOP. 
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The locked same-origin policies must also be incorporated into plugin architectures 
such as Flash, Java, and Adobe Reader. To resist pharming and active attacks, these ar¬ 
chitectures must enforce the locked same-origin policies for network requests and other 
accesses done on behalf of plugins. Also, some plugins architectures allow direct socket 
access, and the locked same-origin policies cannot govern network requests made using 
direct socket access. We consider it the Web site’s responsibility to provide appropriate 
protections when direct socket access is used. 

The locked same-origin policies do not address problems in the Javascript language 
or implementation (e.g.. Javascript Prototype Hijacking) [92], cross-side scripting (XSS) 
vulnerabilities in servers, and cross-site request forgery (XSRF) attacks. Other research 
efforts address XSS vulnerabilities [43, 51, 74, 79, 107, 137, 138] and XSRF attacks [10, 
60, 63, 64], and these techniques complement our work. We also do not address browser- 
side cross-site scripting vulnerabilities, such as Universal XSS [92], 

7.10 Applications to Web authentication 

In this section, we discuss how the locked same-origin policies can help protect two ex¬ 
isting machine authentication mechanisms, client side-SSL and SSL-only cookies, against 
pharmers and active attackers. However, the Web authentication problem is actually two 
distinct subproblems: the initialization of users’ authentication credentials, i.e., the reg¬ 
istration problem, and the use of those credentials to authenticate users to Web sites. Our 
discussion in this section focuses primarily on the latter; we discussed the registration prob¬ 
lem in Chapter 4. 

7.10.1 Client-side SSL 

Intuitively, since client-side SSL authenticates users with end-to-end cryptography, one 
might expect it would protect sensitive Web sessions against pharming and active attacks, 
but unfortunately, the presence of dynamic pharming vulnerabilities proves this is not the 
case. However, using the strong locked same-origin policy in conjunction with client-side 
SSL results in an authentication scheme with strong security properties. The user is not 
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required to memorize her private key. After the user imports her private key, her browser 
uses it automatically. Although an adversary may be able to trick a user into participating in 
mutual authentication using SSL, the adversary cannot use this interaction to impersonate 
the user at another Web site. Authentication requires knowledge of the private key, which 
the user’s browser always keeps secret. As a result, the browser authenticates the user’s 
requests cryptographically and the strong locked SOP isolates the user’s authenticated ses¬ 
sions from malicious subjects - even if the adversary is a pharmer or active attacker. 

7.10.2 SSL-only cookies 

Many Web sites use cookies for machine authentication. For example, some Web sites 
offer a “remember me” option, which sets a persistent cookie on a user’s machine. The 
browser will present this cookie during subsequent visits to the Web site, enabling the user 
to bypass the initial login process. Some existing anti-phishing solutions also use machine 
authentication cookies to complement regular password authentication. Examples include 
Bank of America’s SiteKey [9] and similar approaches by ING Direct [52], Vanguard [123], 
and Yahoo [139]. In current browsers, cookie authentication resists phishing attacks but is 
vulnerable to pharming attacks. Our locked same-origin policies protect SSL-only cookies 
against pharmers and active attackers. Thus, in conjunction with browsers enforcing a 
locked SOP, Web sites can use SSL-only persistent cookies to authenticate users and resist 
phishing, pharming, and active attacks. 

7.10.3 Other authentication mechanisms 

The locked same-origin policies nicely complement other authentication mechanisms 
designed to resist pharming, such as Phoolproof phishing prevention [93] and Passpet [142], 
Our policies also help these schemes resist dynamic pharming attacks. For more informa¬ 
tion on these mechanisms, see Chapter 8. 
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Chapter 8 
Related work 


8.1 Anti-phishing mechanisms 

Several anti-phishing mechanisms help provide information to users regarding the trust¬ 
worthiness of Web sites. Since studies have shown that users can be fooled by misleading 
domain names and do not understand browser security indicators [24, 25, 35, 37, 49, 56, 
108, 130, 135], several researchers and security vendors have developed browser exten¬ 
sions to make it easier for users to interpret relevant security information [17, 49, 78, 114], 
use a blacklist to help identify known phishing sites [27, 88], or establish trusted paths with 
sites users have a relationship with [23, 136, 140]. Recent versions Firefox and Internet 
Explorer have adopted similar mechanisms. However, these approaches still expect some 
degree of diligence from users to reliably observe security warnings and indicators to op¬ 
erate securely, and studies have shown that users still have troubling interpreting improved 
security indicators and warnings [56, 108, 135, 136]. In addition, studies have also shown 
that many browser extensions which try to automatically detect phishing sites are often 
wrong and inconsistent [145]. 

8.2 Phishing and pharming resistant authentication 

Another approach to resisting phishing attacks is better password management. Pass¬ 
words are still the dominant method of Web authentication. Password databases included 
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with most modern Web browsers automatically bll in passwords for users. However, users 
might still manually disclose their passwords to phishing sites or use the same password 
for multiple sites. Password hashing addresses these problems by hashing the user’s secret 
password together with a variable, non-secret string (e.g., each site’s domain name) to pro¬ 
duce per-site passwords [1, 36, 47, 71, 72, 104, 142]. Recent work in this area has made 
usability one of the primary goals [47, 104, 142], but studies have shown some users still 
have trouble using them correctly and securely [15]. Also, if a password hashing scheme 
generates passwords based on the site’s domain [47, 104], it is vulnerable to pharming at¬ 
tacks. Passpet [142] provides some resistance to pharming attacks, but is still vulnerable to 
dynamic pharming. 

The Phoolproof phishing prevention system uses cell phones to manage client-side SSL 
certificates for authentication on behalf of users [93]. In Phoolproof, a user logs in using 
a secure bookmark on her cell phone. The cell phone then 1) initiates an SSL connection 
to the Web site via a Bluetooth connection with a Web browser on the user’s computer; 2) 
checks the site’s X.509 certificate against the one stored in the bookmark; and 3) authen¬ 
ticates the user via client-side SSL. Although Phoolproof verifies the site’s certificate in 
step 2, this protocol is still vulnerable to a dynamic pharming attack if the adversary is able 
to pharm the user (i.e., serve the user a Web page which appears to come from the target 
domain) before she activates the login process. 

ForceHTTPS is a browser policy designed to help resist pharming and active attacks by 
making certificate errors less ambiguous to the browser [54], When a Web site opts in to 
ForceHTTPS, the browser does not allow users to override certificate errors for the site’s 
domain and refuses to import non-HTTPS libraries for the site’s documents. ForceHTTPS 
stores the “opt-in” information in a cookie. Since, like all cookies, ForceHTTPS cookies 
can be used to track users, ForceHTTPS allows users to clear ForceHTTPS cookies, after 
which time users are vulnerable to pharming attacks until the site can reset the cookie. 
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8.3 Key Continuity Management and applications 

The locked same-origin policies were inspired by the concept of Key Continuity Man¬ 
agement (KCM), a model for key management first proven successful by SSH [102, 143] 
and later made more explicit by Gutmann [45]. KCM associates public keys with subjects 
and takes defensive action when a subject’s public key unexpectedly changes. Garfinkel 
expands on KCM further, and applies it to S/MIME [39]. 

The locked same-origin policies are similar to work done independently and concur¬ 
rently by Masone et al. on Web Server Key Enabled Cookies, a new cookie policy inspired 
by KCM that tags SSL-only cookies with the server’s public key and allows access only to 
a server which can authenticate itself to the same key [76]. However, their proposal falls 
short of protecting cookies against dynamic pharming attacks. Also, they do not address 
pharming attacks against other Web objects or other Web authentication mechanisms, e.g., 
client-side SSL, nor do they address subdomain object sharing or key updates. 

8.4 DNS rebinding 

Security researchers and browser developers have been aware of DNS rebinding vul¬ 
nerabilities since as early as 1996 [42], In 2001 and 2002, Jim Roskind and Adam Megacz, 
resp., described firewall circumvention DNS rebinding attacks using DNS records with 
short TTLs [77, 103]. DNS pinning was adopted by browsers to defend against these kinds 
of attacks, but pinning has a lengthy and controversial history in Firefox and Mozilla [82, 
83, 84]. The current implementation is an explicit compromise to support dynamic DNS 
and round robin DNS for failover. In August 2006, Martin Johns discovered a reliable 
technique for circumventing DNS pinning completely [61], and in early 2007, Johns and 
Kanatoko found additional DNS rebinding vulnerabilities with Flash and Java [62, 67], 

Jackson et al. present a comprehensive analysis of DNS rebinding vulnerabilities, in¬ 
cluding issues with Flash, Java, VPNs, caching, and proxies [55]. They discuss several 
countermeasures, including host name authorization, a technique based on a variant of re¬ 
verse DNS lookups. With cooperation from Web sites’ DNS servers, host name authoriza¬ 
tion enables clients to determine the valid set of domain names for a particular IP address. 
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Host name authorization is promising approach, but since it relies on DNS, it is ineffective 
against adversaries capable of subverting DNS. 

8.5 Email for authentication 

Other researchers have proposed leveraging email for authentication [3, 8, 38, 44, 122]. 
In particular, the design of Simple Authentication for the Web (SAW) by Horst and Sea- 
mons is similar to our email registration ceremony [122], The main difference is that we 
propose using email only for relatively infrequent machine registrations, i.e., credential ini¬ 
tialization, while the SAW authors propose using email authentication as a direct replace¬ 
ment for passwords. In SAW, users receive a fresh email link during each authentication 
attempt. Also, the SAW authors do not consider social engineering attacks that try to steal 
authentication links. 


8.6 Studies that attack users 

Security researchers have conducted a number of studies that simulate attacks against 
users. Several studies have tried to evaluate how well individuals can identify phishing 
emails and pages [24, 56, 135]. However, these studies do not fully address the design 
issues we identified in Section 5.1. They were all conducted in a laboratory environment, 
and the users were either told the purpose of the experiment or asked to role-play a fictitious 
identity. 

To help create the experience of risk, some laboratory studies have employed decep¬ 
tion and required users to participate with their own accounts. Egelman et al. conducted 
such a study to evaluate the effectiveness of browser phishing warnings [28]. Users made 
purchases with their own credentials, and the researchers sent the users spear phishing 
emails related to those purchases which triggered phishing warnings in Firefox and Inter¬ 
net Explorer. Schecter et al. asked real Bank of America SiteKey customers to log into 
their accounts from a laptop in a classroom [108]. Although SiteKey uses challenge ques¬ 
tions, Schecter et al. did not evaluate SiteKey’s use of them. Instead, they focused on 
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whether each user would enter her online banking password in the presence of clues indi¬ 
cating her connection was insecure. They simulated site-forgery attacks against each user 
by removing various security indicators (e.g., her personalized SiteKey image) and caus¬ 
ing certificate warnings to appear, and checked if each user would still enter her password. 
Since SiteKey will only display a user’s personalized image after her computer is regis¬ 
tered, Schecter et al. first required each user to answer her challenge questions during a 
“warm-up” task to re-familiarize her with the process of logging into her bank account. No 
attack was simulated against the users during this task. 

Requiring users to use their own accounts is certainly a good start for creating a sense of 
risk, but the degree to which the academic setting of the physical location of these studies 
affected the users’ evaluation of their actual risk is unclear. Even if the experimenters were 
not in the same room as the users while they used the computer, the fact that they were 
nearby may have influenced the users to appear “helpful” and behave with less caution 
than they normally would. 

A few studies have simulated attacks against users in the field without obtaining prior 
consent. One study at the United States Military Academy at West Point sent cadets a sim¬ 
ulated phishing email from a fictitious Colonel “commanding” them to click on a link [31]. 
Studies by Jagatic et al. [57] and Jakobsson et al. [58] also remotely simulated phishing 
attacks against users. Although these studies closely simulated real attacks, provided large 
data sets, and achieved a high level of ecological validity, the absence of prior consent 
raises ethical issues. After learning that they were unknowing participants in one study, 
some users responded with anger and some threatened legal action [19]. Also, these stud¬ 
ies collected only a limited amount of demographic and behavioral data and did not conduct 
a exit survey to probe users’ decisions. 

8.7 User conditioning and education 

Previous anti-phishing research has attempted to take advantage of user conditioning 
by using secure attention keys. Two anti-phishing tools, PwdHash [104] and Web Wal¬ 
let [136], employ a secure attention key to create a trusted path between the user and the 
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browser. Although these tools require users to activate the secure attention key before en¬ 
tering any sensitive information, they may be vulnerable to attacks which persuade users to 
omit the SAK (Section 3.3.1). A user study of Web Wallet suggests that this attack strategy 
can be effective [136]. 

Related to conditioning is training and education. Several researchers have proposed 
innovative educational methods for teaching users about Internet security and social en¬ 
gineering attacks [73, 111, 132]. Their initial results are promising, and related research 
suggests that users who better understand Internet risks may be more likely to resist at¬ 
tacks [26]. However, user education may have its limitations. If education is not period¬ 
ically reinforced, satisficing users may forget or omit defensive habits they have learned. 
Also, a study consisting of interviews designed to reveal users’ decision making strate¬ 
gies for suspicious emails suggests that while users may be able to manage risks they are 
familiar with, it can be difficult for them to generalize this knowledge to resist unfamiliar 
attacks [25]. These results suggest that educational approaches may require continual adap¬ 
tation to address new attacks; otherwise users' defensive strategies may become outdated 
and ineffective. 
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Chapter 9 
Conclusion 


Over the last decade, social engineering attacks on Internet, such as phishing, have 
grown considerably, and we anticipate human factors will remain one of the most important 
and challenging aspects of computer security for the foreseeable future. Although users are 
often considered the weakest link in computer security, few security mechanisms address 
this problem constructively. To remain safe, many current ceremonies burden users to 
detect attacks and refrain from actions they often instinctively perform to complete routine 
tasks, e.g., entering their usernames and passwords. 

Our user study results suggest that 1) ceremonies can affect user behavior, for better 
or worse, and 2) the resiliency of a ceremony to social engineering is related to whether 
the actions it conditions users to take are safe to perform in the presence of an adversary. 
These results suggest that conditioned-safe ceremonies may be a useful notion for building 
user-centric ceremonies that resist social engineering attacks. We proposed several design 
principles for conditioned-safe ceremonies and described one ceremony, email registration, 
designed according to these principles. Although email registration may be an imperfect 
approximation of what we would ultimately like out of a conditioned-safe ceremony, we 
believe it is nonetheless a useful example for exploring and evaluating this notion further. 

Stronger social engineering threats, e.g., pharming, have also become more visible 
over the last decade, and it vital that we develop mechanisms to resist these attacks. We 
demonstrated how adversaries can use dynamic pharming attacks to hijack users’ authen¬ 
ticated sessions in current browsers, irrespective of the authentication mechanism. Dy- 
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namic pharming enables an adversary to eavesdrop on sensitive content, forge transactions, 
sniff secondary passwords, etc. To address dynamic pharming attacks, we introduced two 
locked same-origin policies, which enforce access control using servers’ X.509 certificates 
and public keys, rather than domain names. We evaluated the security and deployability of 
our approaches and showed how browsers can deploy these policies today to substantially 
increase their resistance to pharming attacks and provide a foundation for the development 
of pharming resistant authentication ceremonies. 

The fact that 42% of email users in our study were vulnerable to our simulated attacks 
exemplifies the formidable challenge in designing ceremonies to resist social engineering 
attacks. However, we are hopeful that the future deployment of user-centric mechanisms 
like conditioned-safe ceremonies and the locked same-origin policies will help tip the ad¬ 
vantage away from malicious elements on the Internet. 
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Appendix A 

User study exit survey questions 


1. What is your UCB Movie Predictions login name? 

2. What is your gender? 

Female 

Male 

3. What is your age? 

18-21 

22-25 

26-30 

31-40 

41-50 

51-60 

60 and over 

4. Please choose one of the following: 

Undergraduate student 
Graduate student 
Non-student, please describe: 
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5. If you are a student, please enter your major area 

Economics or business 

Social sciences (i.e., psychology, sociology, etc.) 
Computer science 

Chemistry, physics, biology, or other natural sciences 
Humanities (i.e., literature, languages, history, etc.) 
Engineering 
Other (please specify) 

6. What operating system do you primarily use? 

Windows 
Linux 
Mac OS 

Other (please specify) 

7. What Web browser do you primarily use? 

Internet Explorer 

Firefox 

Opera 

Safari 

Other (please specify) 

8. How many hours a week do you use a Web browser? 

0-5 

5-10 

10-20 

20 + 



123 


9. What kind of financial transactions do you conduct online? 

Auctions (e.g., Ebay) 

Banking 

Investing (e.g., stocks and mutual funds) 

PayPal (and other money transfer services) 

Shopping 

Other (please specify) 

10. Aside from this study, how long have you conducted financial transactions online? 
Never 

Less than six months 
Six months to a year 
One year to two years 
Over two years 

11. What are your biggest security concerns when browsing the Web? Enter up to three 
concerns. 

12. Briefly describe what precautions, if any, you may take when logging into a Web site. 
Please feel free to list precautions you only sometimes take or take only at certain sites. 
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13. In the previous question, you were asked to list what precautions you may take when 
you log into a Web site. In this question, we would like you to evaluate how often and 
thoroughly you apply these precautions when logging into different types of Web sites. 

Site types: 

a shopping Web site? 

a social networking site (e.g., Facebook, MySpace)? 

PayPal? 

Web email (e.g., Gmail, Hotmail)? 

UCB Movie Predictions (the Web site for this study)? 

Choices for each site type: 

“I rarely take the precautions I listed when logging into this type of Web site” 

“I sometimes take the precautions I listed when logging into this type of Web site” 
“I usually take the precautions I listed when logging into this type of Web site” 

“I always take the precautions I listed when logging into this type of Web site” 

“I don't use this type of Web site” 

14. What was average amount of time you spent interacting with UCB Movie Predictions 
each time you logged in? 

0-5 minutes 
5-10 minutes 
10-20 minutes 
20+ minutes 

15. During your interactions with UCB Movie Predictions, did you ever see something 
which looked suspicious or dangerous? 

no 

yes - if yes, describe what it looked like and when you saw it. 

16. If you answered yes to the previous question, describe what your reaction was and if 
you did anything, what you did. 
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17. Do you remember seeing the above warning at any point during the study? 
no, I don’t remember seeing the warning 
yes, I remember seeing the warning 

If you remember seeing it, describe how it affected your decisions (if at all) while 
interacting with the study Web site. Please be specific as possible. 


Image shown to challenge question users: 

Warning! To protect the security of your account: 

♦ Do not share your challenge question answers with others. 

♦ Do not answer your challenge questions if you see any security warnings or the web site looks 
suspicious. 

Image shown to email users: 

Warning! To protect the security of your account: 

♦ Never forward the registration email. 

♦ Never not copy/paste any links or information. The only safe action is to click on the link in the 
email. 

18. During the study, you might remember seeing a page similar to the one below. Study it 
for a moment and then answer the next question. 

♦ Challenge question users were shown Figure 5.4(b). 

♦ Forwarding attack emails users were shown Figure 5.6(a) (or a similar text ver¬ 
sion, depending on the attack details - see Section 5.2.4). 

♦ Cut and paste attack emails users were shown Figure 5.6(b) (or a similar text 
version, depending on the attack details - see Section 5.2.4). 


If you followed the above instructions to forward the registration email, explain why. If 
you chose not follow the instructions, explain why not. If you don’t remember this page or 
what you did, tell us what you don’t remember. 


19. UCB Movie Predictions used <challenge questions/email registration> to help make 
your logins more secure. Have you used cchallenge questions/email registration> before 
at other Web sites? 
no 


yes - if yes, which Web sites? 
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20. Briefly describe how you think registration using <challenge questions/email> works 
and what, if any, security benefits it has. 

21. Rate how safe you would feel using a Web site which uses the following different login 
mechanisms: 

Login mechanisms: 

Password for logins + no registration 

Password for logins + <challenge questions/email> for registration 

Answer choices: 

Not secure at all 
Somewhat secure 
Fairly secure 
Very secure 
I don't know 

22. Rate the convenience of each of the following login mechanisms: 

Login mechanisms: 

Password for logins + no registration 

Password for logins + <challenge questions/email> for registration 

Answer choices: 

Nearly impossible or very annoying 
Hard or slightly annoying 
I could get used to it 
I hardly noticed it 
I don’t know 

23. Did you find anything difficult or annoying about this study? 

24. Did you feel engaged/interested during the entire duration of the study? 
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25. Do you have any general comments about this study? 



128 


Appendix B 

User study exit survey responses 






























































































































































































































































































































Group 1 (challenge questions): Demographics 
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223 phishing, viruses i usually always type the original Always Always Sometimes Always Rarely Sometimes 

website (no clicking on links) and 
look for my passkey if there is one 

228 Identity Theft No response Always Always Sometimes Usually Sometimes Usually 
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entered answers no N/A yes: I didn’t take too much pre- I answered it because it was required in order 

caution even after seeing it. I just to log in, otherwise there was no way for me 
thought I would trust it since this to enter, 
is part of UC Berkeley’s experi¬ 
ment. 



















































































































me that the security same message came 
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Group 1 (challenge questions): Registration attitudes 
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Group 1 (challenge questions): Registration attitudes 
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one question, but if someone knows 
the answers they can get your pw right 

























































Group 1 (challenge questions): Registration attitudes 
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check because you are able to check if 
you are being asked the right security 
questions and if you know the correct 
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Group 1 (challenge questions): General questions/feeback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

73 

5-10 minutes 

Yes, I found the security questions an¬ 
noying. 

yes 

Had I been on a private, personal com¬ 
puter that I had registered before, I 

would have known not to answer the ex¬ 
tra questions. As it is, I am using a pub¬ 
lic computer and so expected to have to 

answer security questions. 

78 

0-5 minutes 

No 

yes 

Well done! I was so focused on making 

money from the predictions that I failed 

to take any security precautions 

83 

0-5 minutes 

No, it was easy to follow. 

yes 

No response 

88 

0-5 minutes 

No response 

yes 

No response 

93 

0-5 minutes 

no 

yes 

No response 

98 

0-5 minutes 

no 

yes 

no 

103 

0-5 minutes 

no 

yes 

No response 

108 

5-10 minutes 

No 

yes 

None 

113 

5-10 minutes 

Only when I was wrong! 

yes 

Great website design - easy to use and 

it looks good. Study was easy to access 

and complete. 

118 

0-5 minutes 

No, not really. Well done. However, I 

feel like I missed something that was 

supposed to indicate a security com¬ 
promise 

yes 

No response 

123 

5-10 minutes 

None 

yes 

None 

128 

5-10 minutes 

No 

yes 

I kept forgetting to do my movie pre¬ 
dictions - I think there should be a re¬ 
minder email everyday. 

133 

5-10 minutes 

no 

yes 

I enjoyed this study, it was very inter¬ 
esting. 

138 

5-10 minutes 

No 

yes 

No response 

143 

5-10 minutes 

No 

yes 

No response 

148 

0-5 minutes 

No not really. 

yes 

No response 

153 

5-10 minutes 

No, it was nice. :) 

no 

The debriefing really took me by sur¬ 
prise. I was completely unable to guess 

the hypothesis. Kudos! 

158 

5-10 minutes 

Sometimes it was hard to remember to 

predict the movies every day. 

yes 

No. 

163 

0-5 minutes 

no 

yes 

no 

168 

5-10 minutes 

no 

yes 

interesting study, thanks! 

173 

0-5 minutes 

the deception was a little annoying - i 

was enjoying predicting which movies 

would do well! 

yes 

nice one, guys. 

178 

0-5 minutes 

no 

yes 

No response 
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Group 1 (challenge questions): General questions/feeback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

183 

0-5 minutes 

No. 

yes 

I liked trying to figure out the winning 

movies! 

188 

No response 

No response 

No response 

No response 

193 

0-5 minutes 

I watch movies once a year. I basically 

took a poll from my friends before I 

answered the questions. 

no 

No response 

198 

5-10 minutes 

I was frustrated that the box office re¬ 
sults were slow to update. 

yes 

I still don’t understand how the attack 

against me was supposed to work. Does 

it assume that the actual site’s server has 

been compromised? 

203 

5-10 minutes 

No response 

No response 

No response 

208 

0-5 minutes 

No 

yes 

I like the twist! =) 

213 

No response 

No response 

No response 

No response 

218 

5-10 minutes 

Yeah! The results for top gross¬ 
ing movie for a particular day would 

switch over time, decreasing my bonus 

$! 

yes 

No response 

223 

0-5 minutes 

No response 

yes 

No response 

228 

0-5 minutes 

Exit survey is kind of long. 

yes 

Fun to predict movies. 

233 

0-5 minutes 

No. 

yes 

Clever. 

238 

5-10 minutes 

No response 

yes 

No response 

243 

5-10 minutes 

no 

yes 

no 

248 

0-5 minutes 

no 

yes 

it was a good study, there was lots to 

interpret from the information 

253 

5-10 minutes 

I didn’t fully understand that the order 

of predictions makes a difference until 

after the 3rd try because I didn’t fully 

read the FAQ until then. 

yes 

No response 

258 

0-5 minutes 

no 

yes 

no 

273 

5-10 minutes 

No, not really. 

yes 

No, except that the login part at the end 

was a little weird. I was a bit hesitant 

on entering the site. 

278 

5-10 minutes 

nope 

yes 

The movie prediction process was al¬ 
luring and distracting from the true mo¬ 
tives of the study. 

283 

0-5 minutes 

No, just now I am very cautious about 

answering these questions. 

yes 

No response 
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Group 2 (email with warnings, forwarding attack): Demo¬ 
graphics 
































































































































































































Group 2 (email with warnings, forwarding attack): Web se¬ 
curity attitudes 
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browser, watch for bad certificates 

























































































Group 2 (email with warnings, forwarding attack): Web se¬ 
curity attitudes 


57 



used by many people. 
























































































































































Group 2 (email with warnings, forwarding attack): Web se¬ 
curity attitudes 
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counts hacked with fake websites the urls when I’m logging in to 








































































'up 2 (email with warnings, forwarding attack): Study 
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up 2 (email with warnings, forwarding attack): Study 
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was the easiest path regardless. The email address looked legitimate enough, 
forwarded url no N/A no because that is what it said to do 

























































































































































up 2 (email with warnings, forwarding attack): Study 
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up 2 (email with warnings, forwarding attack): Study 


66 



group for ucb movie predictions- 
because I wasn’t quite sure what 
was going on. But after compar¬ 
ing the two, I went ahead and for¬ 
warded the e-mail. 


























































Group 2 (email with warnings, forwarding attack): Registra¬ 
tion attitudes 
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Group 2 (email with warnings, forwarding attack): Registra¬ 
tion attitudes 
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noying 

yes: Don’t remember. To make sure that the e-mail account Fairly secure Very secure I hardly noticed it I could get used to it 

really exists. 


































































Group 2 (email with warnings, forwarding attack): Registra¬ 
tion attitudes 
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Group 2 (email with warnings, forwarding attack): Registra¬ 
tion attitudes 
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site. This increases the verification of 
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Group 2 (email with warnings, forwarding attack): General 
questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

77 

No response 

No response 

No response 

No response 

87 

No response 

No response 

No response 

No response 

92 

No response 

No response 

No response 

No response 

97 

0-5 minutes 

nope 

yes 

none 

102 

0-5 minutes 

No 

yes 

No response 

107 

5-10 minutes 

No. 

yes 

No response 

112 

0-5 minutes 

When the predictions didn’t update on 

Sunday. I wanted my money. My fi¬ 
ance and I were actually secretly com¬ 
peting. 

yes 

NOOOOOOOO 

117 

5-10 minutes 

Nope. 

yes 

Clever! I wouldn’t have guessed the 

real nature of the study until that bogus 

email forwarding request made me sus¬ 
picious 

122 

5-10 minutes 

no 

yes 

I really did enjoy it! 

127 

0-5 minutes 

no 

no 

No response 

132 

0-5 minutes 

(minor) subscribing to paypal 

yes 

tricky 

137 

0-5 minutes 

no 

yes 

No response 

142 

0-5 minutes 

the fake ’’pharming attack” was a little 

annoying 

yes 

nope 

147 

0-5 minutes 

slow movie updates 

yes 

no 

152 

0-5 minutes 

no 

yes 

No response 

157 

5-10 minutes 

no 

yes 

the security situations seemed a lit¬ 
tle unrealistic (don’t cut and paste the 

URL?) 

162 

0-5 minutes 

I was unsure whether you could regis¬ 
ter and participate from more than one 

computer 

yes 

No response 

167 

0-5 minutes 

No. 

yes 

No response 

172 

5-10 minutes 

The movie rankings were sometimes 

delayed. 

yes 

No response 

177 

0-5 minutes 

No response 

yes 

No response 

182 

5-10 minutes 

remembering to sign in 

yes 

No response 

187 

0-5 minutes 

No 

no 

No response 

192 

5-10 minutes 

no 

no 

No response 

197 

5-10 minutes 

no 

yes 

no 
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Group 2 (email with warnings, forwarding attack): General 
questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

202 

0-5 minutes 

Having to set up a paypal account 

and the ucb movie prediction account. 

Also, I did not like that the movies to 

choose from were the same throughout 

the whole experiment. 

no 

I think the fake reason for the experi¬ 
ment was too obvious given what we 

were doing. It might have been better 

to be more vague or misdirecting there, 

so there is no chance people think there 

is a different reason for the experiment. 

207 

5-10 minutes 

No 

yes 

No response 

212 

5-10 minutes 

Nope, not especially 

yes 

Tricky tricky. 

217 

No response 

No response 

No response 

No response 

222 

5-10 minutes 

No, not really. Even the whole ’’reg¬ 
ister this computer” thing wasn’t too 

bad. 

yes 

Well done! I love movies so I was com¬ 
pletely oblivious to the ulterior motives 

of the study. 

227 

5-10 minutes 

Not at all 

yes 

The inclusion of movies made it more 

appealing for me 

232 

0-5 minutes 

no 

yes 

No response 

237 

5-10 minutes 

no 

yes 

no 

242 

No response 

No response 

No response 

No response 

247 

5-10 minutes 

Registering each computer. 

no 

No response 

252 

0-5 minutes 

constant timeouts (both my computers 

are relatively secured from other users 

besides me) 

yes 

I study social psychology and this study 

fooled me pretty well. Great job, guys! 

257 

0-5 minutes 

not at all 

yes 

Great turn-around purpose of the study. 

Very intriguing. 

267 

5-10 minutes 

It ask me to register a few times. I 

don’t know if it is because I didn’t read 

the instructions or if it was because of 

an error. 

yes 

Predicting movies was kinda fun. It’s 

like gambling without losing money. 

272 

0-5 minutes 

no 

yes 

no 
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Group 2 (email with warnings, forwarding attack): General 
questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

277 

5-10 minutes 

Well I don’t feel like I understand what 

the purpose was of the ’movie predic¬ 
tions’ bit of this study...was it related 

to it at all or was it all fake and it was 

just a security breach simulation via 

internet? I wasn’t expecting this at all, 

really. 

no 

I feel more lost and confused about 

what happened in this study than en¬ 
gaged... that debriefing was way too 

long and inundated with strange infor¬ 
mation... may be I need to read it over 

more slowly but I think this sudden 

change in experimenting needs to be 

better illuminated. Plus, not that many 

people are familiar with internet hack¬ 
ing and how to detect it so I suspect a lot 

more people in this study are even more 

lost than I am about what this study was 

actually about. 

282 

0-5 minutes 

No 

yes 

No response 



















Group 3 (email without warnings, forwarding attack): De¬ 
mographics 


76 



Auctions, PayPal 

Female 18-21 Undergraduate Natural sciences Windows Internet Ex- 10-20 Banking, Shopping, Never 

plorer PayPal 

Female 22-25 Graduate Optometry Windows Internet Ex- 20+ Banking, Shopping 6-12 r 

plorer 













































































































































Group 3 (email without warnings, forwarding attack): De¬ 
mographics 


177 


©P 

a 

04 

73 



























© 

09 



























-© 

S3 

73 



























04 

u 

S- 

.g 





•S 

0 





■S 









-S 

0 





a 


S3 

V 

J3 

a 

o 

c3 

44 


c3 

44 

Vh 

3 

44 

o 

£ 



i 

0 ) 


3 

o 

£ 


c3 

<u 

c3 

o 


Vh 

3 

(D 


c3 

<u 


o 

£ 

i 

4) 

Vh 

3 

4) 


c3 

4) 

3 

0 

£ 


3 

s 

in 

So 


So 

too 

04 



So 



So 

So 


too 


too 


04 

So 

too 


So 


&. 

73 

S 

04 


04 

04 




04 


SO 


04 

04 


<N 


04 



04 

04 


04 

a 


a 

S3 

<C3 

O 

A 


A 

1 

1 

SO 



1 


V 


A 

A 


1 


A 


1 

SO 

1 

1 


A 

V 


«3 



top 




top 

in 


top 





top 




so 




top 



top 


o 



3 




3 

S3 


3 





3 




3 




3 



3 





’a 




a 

.2 


’a 





’a 




a 




’a 



’a 


u 

3 

z 

S3 

3 

S- 

04 

# g 


a 

o 

a 

00 

13 

a 

so 

73 

a 

top 

S3 

'H 

& 

-S3 

00 

13 

a 

too 

73 

a 

§3 

l 

tj 

3 

< 


a 

o 

a 

00 


13 

a 

too 

3 

a 


top 

3 

’a 

B- 

a 

00 

a 

o 

a 

00 

a 

SO 

3 

a 

top 

3 

’a 

& 

a 

00 


top 

.3 

aj 

^3 


13 

a 

So 

3 

a 

top 

3 

’a 

8* 

a 

00 

a 

0 

a 

00 


top 

3 

’a 

B- 

a 

00 

a 

0 

a 

00 

13 

a 

So 

3 

a 

"3 



„ 

c/T 



„ 

top 


„ 





„ 

in 





in 


, 



„ 

in 


a 


top 

0 

top 

top 

top 

3 


top 


top 


top 

top 

0 

top 


top 


0 

top 

top 


top 

top 

0 

*s 

© 


3 

o 

S3 

S3 

3 


73 

3 

13 

a 

3 


3 

3 

o 

3 


3 


o 

3 

3 


3 

3 

O 

S3 

CS 

04 




3 

a 

3 

a 

a 

3 

a 

a 

a 


a 

a 




3 



a 

a 

3 

a 

’2 

a 


ff 

S3 


3 

o 

S3 

3 

3 

o 

So 

3 

;o 

3 


3 

3 

o 

3 


3 


o 

3 

3 

>, 

3 

3 

O 


o 


3 

3 

73 

73 

3 

a 

3 

3 

3 

3 


3 

3 

3 

3 


3 

13 

3 

3 

3 

3 

3 

3 

3 

a 

-© 


CQ 

< 

CQ 

CO 

a 

00 

a 

a 

a 

a 


a 

a 

< 

a 


a 

a 

< 

a 

a 

a 

a 

a 

< 


a 

V 

04 




























£ 


o 


O 

o 

O 



O 


o 


o 



o 





o 



0 



■§ 


03 

04 


Ot 

04 

04 



04 


04 


04 

+ 


04 


+ 


O 

04 

+ 


04 

+ 


& 


a 

O 


O 

O 

O 



O 


O 


O 

o 


O 


o 



O 

0 


O 

0 


© 


1 




' 1 



’ 1 




1—1 

04 


1—1 


04 


in 

r_ * 

04 


1 

04 














X 







X 





















a 







a 









so 

Si 



























u 

73 

g 

o> 


X 


X 

X 




X 




X 

X 


X 


"5 


X 


X 


X 

X 


i 


£ 


£ 

£ 

1 



£ 


£ 

<3 


£ 


«a 


£ 

o 

& 

1 

<2 



<2 



0 


44 


<u 

<u 



D 


2 

•— 

0) 

0) 


<u 


2 

•— 

4) 

4) 


4) 

4) 


*E 

•_ 


S—i 


_s- 

_s- 

3 



J-i 



o 

# v- 

S-H 


_v- 



o 

_S-< 

3 

# Vh 


_v- 

S-H 


a 

.© 


Uh 


a 

a 

00 



a 


3 

'Hh 

a 

a 


a 


3 

'Hh 

a 

00 

a 


a 

a 














in 









cz, 










£ 

o 


£ 

o 

£ 

o 

00 

o 



o 


% 

O 


00 

O 

£ 

o 


00 

O 


00 

o 


& 

O 

00 

o 

0 


0 

0 




73 


73 

73 

o 



T3 


TD 


o 

T3 


o 


o 


T3 

o 

73 


73 

73 


o 



3 


S3 

3 

3 



3 


3 


3 

3 


3 


3 


3 

3 

3 


3 

3 





& 


? 

t? 

s 



& 


5 


S 

& 


S 


s 


5 

s 

s 


? 

? 














in 





So 








in 















<D 





top 








U 















3 













3 




















O 





















O 


3 

a 


<$ 

s 


a 




§3 

on 

czi 


3 

a 








44 





o 

3 


g 


o 

3 

O 

3 



Vh 

GJ 



4) 

3 

4) 

4) 

4) 

4) 


g 






top 


top 




2 




2 

2 




top 


2 

3 

C 








.3 


3 

.3 

•2h 



‘o 


o 


‘o 

‘o 



3 

OJ 

.3 


‘3 

.2 

.2 


0 



(ZJ 



‘C 

44 

44 

C 


44 

0) 

a 

'C 

<u 

o 

.3 

"3 

3 

s 

p 



3 

5-H 

3 


£ 

o 

3 


3 

1- 

3 

13 

S-H 

3 


jj 

a 

a 

H 

Q 

"C 

<D 

D 

_3 


3 

i-H 

3 

‘o 

"3 

’3 


£ 

0 

3 

top 

_o 


OJ 



'top 

3 


o 

t- 

’top 

3 





O 

o 


"c3 

e3 


o 

Z 

’top 

3 


H 

‘o 

0 

‘3 

0 


0 

4) 

"o 


Q 



W 


< 

a 

X 





a 


£ 

Z 


Q 

< 

a 


£ 

00 

00 


a 

a 





qj 


aj 

£ 

3 



■2 


2 


2 

2 


2 


2 


2 

2 

2 


2 

2 


3 

g 


3 


3 

3 

3 



3 


3 


3 

3 


3 


3 


3 

3 

3 


3 

3 


•£3 

0 




3 

3 

3 



3 


3 


3 

3 


3 


3 


3 

3 

3 


3 

3 


in 

0 


•a 


73 

73 

73 



X5 


T3 


•o 

T3 


T3 


T3 


73 

73 

73 


73 

73 



O 


3 


73 

73 

3 



3 


3 


3 

3 


3 


3 


3 

3 

3 


3 

3 


3 



top 


top 

top 

top 



top 


top 


top 

top 


top 


top 


top 

top 

top 


top 

top 


04 

Q 

S3 

t- 

a) 

-a 


Vi 

44 

73 

Ui 

<D 

73 

Vh 

44 

-a 



Sh 

<D 

•o 


*-> 

<D 

"O 


Sh 

4) 

•a 

Sh 

D 

•a 


u> 

D 

73 


Uh 

4) 

~a 


Sh 

4) 

73 

Sh 

4) 

73 

ui 

4) 

73 


Sh 

4) 

73 

Sh 

4) 

73 


■o 


© 











S3 

in 

C3 

3 


3 

3 

3 



3 


3 


3 

3 


3 


3 


3 

3 

3 


3 

3 


tfl 

3 

a 

P 


D 

D 

P 



P 


P 


P 

P 


P 


P 


P 

P 

P 


P 

P 



















uo 


| 






t 

_ _ | 


Age 



04 


04 

04 

04 



04 


04 


04 

04 


04 


04 


04 

04 

04 


04 

04 




OO 


OO 

OO 

00 



00 


OO 


OO 

OO 


04 


OO 


00 

00 

OO 


00 

GO 


















04 


1—1 


1—1 

1—1 



1—1 

1 


Si 





to 


jj 





u 


_2 



O 








JJ 

JJ 


04 



JJ 


13 

JJ 

3 



13 


13 


13 



13 


to 


13 


-2 


13 

3 


3 

04 



S 


£ 

<u 

13 

s 

£ 

o 



£ 

D 


a 

<D 


£ 

D 

13 

s 


a 

<u 


3 

S 


a 

4) 

13 

s 

13 

s 


£ 

4) 

£ 

4) 


o 





a 


a 



a 


a 


a 



a 




a 




a 

a 






























Si 



On 


7j- 

ON 




Os 




On 

'Tf 


Os 




Os 


Os 



7f 


V 



lO 


SO 

SO 

O' 



O' 


00 


OO 

On 


os 


O 


o 




04 

CO 


in 



















04 


04 

04 

04 


04 

04 


P 
































































































































































































































































































































































































Group 3 (email without warnings, forwarding attack): Web 
security attitudes 


180 






























































Group 3 (email without warnings, forwarding attack): Web 
security attitudes 
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ica login sites which are alarming, 
but I’ve never been scammed per¬ 
sonally. 

getting my information stolen check for SSL encryption when Don’t use Don’t use Rarely Always Rarely Rarely 

giving my CC information 
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followed html url no N/A no I did not follow the instructions because the 
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Group 3 (email without warnings, forwarding attack): Reg¬ 
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yes: dont know... ... Somewhat secure Very secure I hardly noticed it I hardly noticed it 






































































































































































































































































Group 3 (email without warnings, forwarding attack): Reg¬ 
istration attitudes 


98 



264 yes: Forums No response No response No response No response No response 

266 no: Don’t remember Only seems to have security benefits Fairly secure Fairly secure I hardly noticed it I could get used to it 

but a lot for the site, not the user (prevents bots 

from registering) 

269 yes No response No response Fairly secure I could get used to it I could get used to it 











































































200 

Group 3 (email without warnings, forwarding attack): Gen¬ 
eral questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

74 

5-10 minutes 

Just the fact that I could not log in on 

just any computer. Usually I am at 

the library until pretty late at night and 

it was annoying that I could not use 

the library computer because it was a 

’’public computer” to just quickly log 

into the UCB movie predictions web¬ 
site. 

no 

I thought the whole study was a little 

weird since there was so little informa¬ 
tion on the site on the different movies. 

79 

5-10 minutes 

No 

yes 

No response 

84 

10-20 minutes 

No response 

yes 

I think it would have been interested if 

it lasted a few days longer. I think just 

after a few days I started to get an idea 

of how people behave on weekends ver¬ 
sus weekdays and how that would affect 

their movie-going behavior and I didn’t 

have a chance to test my theory. 

89 

5-10 minutes 

Nope. It was pretty fun. 

yes 

No response 

94 

0-5 minutes 

This little twist of events at the end is a 

bit confusing -1 also have not yet made 

7 predictions, and I am uncertain if I 

will still be compensated. 

no 

No response 

99 

0-5 minutes 

No response 

yes 

No response 

104 

10-20 minutes 

no rss feed 

yes 

No response 

109 

0-5 minutes 

nope 

yes 

nope 

114 

5-10 minutes 

Forgetting when movies debuted. 

yes 

Y’all duped me. Nice. 

119 

0-5 minutes 

No, it was easy to use. 

yes 

None 

134 

0-5 minutes 

No response 

yes 

No response 

139 

0-5 minutes 

the previoous day’s predictions were 

not posted promptly. 

yes 

No response 

149 

5-10 minutes 

no 

yes 

no 

159 

0-5 minutes 

No. 

no 

No. 

164 

5-10 minutes 

no. 

yes 

none. 

169 

0-5 minutes 

No 

yes 

No response 

174 

5-10 minutes 

No, it was simple and easy and well- 

explained in the beginning. 

yes 

Great study. 

179 

5-10 minutes 

No 

yes 

No response 


179 


5-10 minutes 


yes 


No response 
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Group 3 (email without warnings, forwarding attack): Gen¬ 
eral questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

184 

0-5 minutes 

It was annoying that sometimes the 

previous day’s movie results were not 

updated by the time I logged in to 

make another prediction the following 

day. 

yes 

I am surprised that I didn’t follow direc¬ 
tions in terms of forwarding the email. 

189 

5-10 minutes 

Many times the daily gross wasn’t 

posted so I would check a few times 

a day, but I could only do it from my 

computer. 

yes 

No response 

194 

0-5 minutes 

No response 

yes 

No response 

199 

0-5 minutes 

actually yeah, it was annoying and 

threw me off that i had to register my 

computer in the first place. 

yes 

tricky, tricky, tricky, but you got me so 

i guess that means that it’s good you’re 

doing this study, right? thanks! 

204 

0-5 minutes 

No. 

yes 

It reminds me of the book called ’’The 

Wisdom of Crowds” 

209 

0-5 minutes 

No response 

no 

I dont even have TV, so the movie pre¬ 
dictions were totally random 

214 

0-5 minutes 

Nope. 

yes 

Fun! 

219 

0-5 minutes 

registration only 

no 

nope 

224 

0-5 minutes 

no 

yes 

No response 

234 

0-5 minutes 

no 

yes 

it was fun! very realistic 

239 

No response 

No response 

No response 

No response 

244 

0-5 minutes 

No 

yes 

No response 

249 

0-5 minutes 

No response 

yes 

No response 

254 

5-10 minutes 

Not really. The ’’error” happening at 

the same time as the exit survey con¬ 
fused me because I thought it was pos¬ 
sible that something had gone wrong, 

rather than the survey being intention¬ 
ally short and I was hesitant to take the 

exit survey because I assumed at the 

time that that would mess up your data 

if I didn’t do all 7 guesses. But I sup¬ 
pose that is something you want inten¬ 
tionally there so there’s no much you 

can do about it. 

yes 

I think it’s pretty effective 

262 

0-5 minutes 

Sometimes I find it annoying that I’m 

being asked an open-ended questions 

(I’d rather just have choices to make a 

decision about) 

yes 

No 
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Group 3 (email without warnings, forwarding attack): Gen¬ 
eral questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

263 

0-5 minutes 

Having to make a paypal account. 

yes 

No 

264 

No response 

No response 

No response 

No response 

266 

0-5 minutes 

We didn’t get a chance to make as 

much money as I though we could? 

But I guess that was part of the sur¬ 
prise/deception. 

no 

Creative, got me. 

269 

10-20 minutes 

No response 

yes 

No response 

274 

5-10 minutes 

Being tricked is frustrating. Also, this 

study makes you guys seem conde¬ 
scending and overly pedantic 

yes 

It makes no sense for us not to trust 

a new link that sends us a registration 

email from the site with which we are 

registered. A ’’fishy” web link would 

not have access to our email address un¬ 
less the original site disclosed that in¬ 
formation. Should we not trust websites 

linked to Berkeley? 

279 

0-5 minutes 

No response 

yes 

No response 
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Group 4 (email with warnings, cut and paste attack): General 
questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

75 

5-10 minutes 

Nope 

yes 

It was fun and interesting. I really 

thought I was trying to predict movie 

trends. Haha. 

80 

0-5 minutes 

Not really. I guess disappointed it had 

nothing to do with predicting movies, 

haha. 

yes 

The idea of email registration sounds 

inconvenient. Emails often get sent to 

people’s spam, or something just flat- 

out dont go through. Also, they dont 

always arrive promptly, which I imag¬ 
ine would get annoying if you want log 

in and out quickly. 

85 

5-10 minutes 

No response 

yes 

I did not realize we had a limited 

amount of 10 days to get in 7 votes. 

This was not explicitly stated anywhere 

on the UCB movie predictions website 

itself and thus was very very mislead¬ 
ing. 

90 

5-10 minutes 

no 

yes 

No response 

95 

5-10 minutes 

No. 

yes 

No. 

100 

5-10 minutes 

not particularly 

yes 

Haha really tricky...didn’t see that com¬ 
ing that’s for sure! 

105 

0-5 minutes 

no 

no 

didnt have time to do it everyday wish i 

could do it whenever i wanted 

110 

0-5 minutes 

no 

yes 

No response 

115 

0-5 minutes 

Not enough time, no daily reminders 

to visit the site. 

yes 

No response 

120 

5-10 minutes 

This last survey was a bit long, but the 

rest of the study was fine. 

yes 

It was fun and finding out the little twist 

at the end was entertaining. 

125 

0-5 minutes 

Yes, filling out this survey. Especially 

because this has nothing to do with 

movie predictions. 

yes 

What exactly is this study about? 

Movie predictions or web site safety? 

135 

0-5 minutes 

NO 

yes 

No response 

140 

5-10 minutes 

No 

yes 

Very tricky! :P 

145 

No response 

No response 

No response 

No response 

150 

5-10 minutes 

no 

yes 

No response 

155 

0-5 minutes 

NO. There should be more like it. 

no 

Good job. 

160 

0-5 minutes 

no. 

yes 

No response 

165 

5-10 minutes 

nope...ease of entrapment at the end is 

testimony :) 

yes 

No response 

170 

0-5 minutes 

No 

yes 

None 

175 

0-5 minutes 

nothing comes into mind. 

yes 

none. 
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Group 4 (email with warnings, cut and paste attack): General 
questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

180 

0-5 minutes 

no 

yes 

no 

185 

5-10 minutes 

No 

yes 

n/a 

190 

0-5 minutes 

Not at all. 

yes 

I failed :( 

195 

No response 

No response 

No response 

No response 

200 

5-10 minutes 

No response 

yes 

No response 

205 

0-5 minutes 

No 

yes 

What is this for? Is it for security con¬ 
cerns? 

210 

5-10 minutes 

No response 

yes 

No response 

215 

0-5 minutes 

nope 

yes 

I would have liked to have more info on 

the other movie grosses. 

220 

5-10 minutes 

I think I would have forgot about the 

movies if there were not constant re¬ 
minders about the 7 log-ins. 

yes 

Um, if you want to make it more legit 

for the movie study, you shouldn’t al¬ 
low people to change their answer once 

they input or at least don’t let them see 

how previous movies did so they can 

change their answers. 

225 

0-5 minutes 

Not really. 

yes 

No response 

230 

0-5 minutes 

no 

yes 

no 

235 

0-5 minutes 

No, except when I wanted the movie 

information to get in sooner. 

yes 

This study was interesting, we should 

have more like it. 

240 

0-5 minutes 

no 

yes 

good design/idea 

245 

0-5 minutes 

No 

yes 

No response 

250 

5-10 minutes 

no 

no 

I had no idea this was about internet se¬ 
curity, I sincerely believed it was about 

predicting market events. Your methods 

work. 

255 

0-5 minutes 

no 

no 

No response 

260 

5-10 minutes 

No response 

yes 

No response 

270 

5-10 minutes 

no 

yes 

i had a false sense of security because it 

was a berkeley site so i never suspected 

it could be ’’dangerous” and my guard 

was definitely down. 

275 

20+ minutes 

No response 

yes 

No response 

280 

0-5 minutes 

no 

yes 

n/a 













































































































Group 5 (email without warnings, cut and paste attack): De¬ 
mographics 


225 


0£ 

a 

& 

a 



























© 

5/3 



























■d 

a 

a 



























© 

CD 

a 

# a 












X 







X 







a 

"a 

"a 

3 



3 

3 

3 


3 




a 

o 

3 



3 

3 


a 

o 


3 


3 





o 

<u 



ID 

ID 

ID 


d 




£ 

a 



a 

<D 


£ 


<D 


(D 



0> 

a 

5/3 

>> 



>. 

>> 

>> 


>> 


3 


CN 

>> 


3 

>. 

>> 





>> 



D 

a 

a 

CN 



CN 

CN 

CN 


CN 


3 



CN 


3 

CN 

CN 


X 


CN 


CN 



p 

a 

ce 

o 

A 



A 

A 

A 


A 


z 


1 

X 

A 


z 

A 

A 


V 


A 


A 



5/3 

r* 



top 





top 


top 





top 




top 


top 


Dh 


top 

5/T 


o 



a 

a 




a 


a 





a 




a 


a 


o 

13 

a 

a 


*-3 



■a 

.2 




x 


‘S- 





‘B- 




‘B- 


’Bh 


X 

p 

X 

.2 


cd 



© 

3 



top 

© 


d. 





& 




& 


D- 


CO 

>. 

3 

3 


es 

5/3 

a 

a 



1 

a 

< 



a 

‘Bh 

& 



o 

X 

00 





c 

X 

00 

'a 

Oh 

13 

Cl. 

>. 

a 

CL 

a 

.o 

CD 

o 

X 

00 

13 

Oh 

O 

X 

oo 


top 

a 

‘m 

a 

P 

a 


a 

< 


-h- 

© 

G 






X 

00 









a 

CLh 

3 

< 


a 

P 



a 

a 

.o 




"a 



, 

top 




„ 

top 

„ 




top 

, 

5/f 

top 


„ 

5/T 

„ 


P 

3 

„ 

top 



~G 


top 

a 


top 

top 

top 

a 

top 




a 

top 

G 

a 

top 

top 

G 

top 



a 

top 

a 


’«■> 

fl 

O 


a 

‘B- 

"a 

a 

a 

a 

‘p 

a 

'a 



‘Bn 

a 

o 

‘Bh 

a 

a 

O 

a 

"a 

top 

< 

a 

‘B- 

13 

a 

© 


3 

p 

Oh 

M 

X 

X 

P 

M 

P 



P 

M 

- a 

Dh 

X 

M 

- a 

M 

CLh 



X 

p 

P 

# a 

a 

o 


§ 

o 

X 

>> 

a 

a 

a 

a 

a 

§ 

o 

X 

§ 

>. 

a 

o 


o 

X 

§ 

CJ 

3 

o 

x 

a 

a 

3 

CD 

a 

a 

a 

>> 

a 

|g 

a 

3 

o 

X 

>> 

a 

£ 

'd 


P 

00 

p 

CQ 

CQ 

P 

00 

P 

CLh 

Z 


00 

P 

< 

00 

CQ 

P5 

< 

oa 

P 

P 

‘Bh 

p 

00 

P 


© 












a 
















© 

£ 






o 






a 

o 


o 


o 






o 





■§ 

"35 

5/3 

La 

+ 



+ 

CN 

+ 


o 



Dh 

+ 

CN 


CN 

+ 

+ 


+ 


CN 


+ 



& 

5/! 


o 



o 

O 

o 






o 

O 


O 

o 

o 


o 


O 


o 



a 


CN 



CN 


CN 


>n 


z 

5-h 

CN 

1—1 


1 

CN 

CN 


CN 


'~ l ' 


CN 















a 



X 




X 





















a 



P 




P 










Lh 











o 
















h 

a 

£ 

© 


X 



X 

X 

X 




P 


X 

3 



X 

3 


X 


X 


X 



1 


£ 



£ 

£ 

£ 


1 


a 


£ 

£ 

3 

‘C 

£ 

£ 

3 

£ 


a 


£ 




© 


D 



CD 

<D 

(D 





a 

ID 

S-H 

,a 

<D 

ID 

S-H 

<D 


<D 


(D 



*3 

La 


5h 



_c- 

H 

S-i 


a 


o 


.5-1 


o 

a 

5- 


o 

. 5-h 


5h 


Vh 



On 

X 


tin 



p 

p 

P 


00 


Z 


p 

a 

o- 

00 

P 

a 

a 

P 


P 


p 















a 
















CO 



£ 

O 



5/1 

£ 

o 

5/1 

£ 

o 

£ 

c 


00 

O 


a 

o 

Oh 


oo 

O 

% 

o 


00 

O 

5/1 

o 

% 

c 


00 

O 


5/1 

o 


% 

C 





T3 



■a 

T3 

•a 


o 




o 

•a 


CD 

■a 

•a 


CD 


a} 


ai 



O 



a 



a 

a 

a 


a 


a 


a 

a 


a 

a 

a 


a 


a 


a 









5 

£ 



2 


o 

Z 


2 

? 


2 

5 

5 


2 


5 


5 






<d 





© 






3 

















c 





d 






.a 

















a 





3 






a 




5/1 













X) 




iD 

X 


<D 




X 




a 
















iD 

CD 



CD 





a 



CD 





ID 








o 



CD 

a 

a 

_<D 

O 


a 

.© 


a 


o 

o 

a 


top 

a 

.CD 

a 




CD 

a 


(D 






o 



.© 

’3 

© 


‘3 


a 


o 

CD 


.a 

’3 

a 




.a 


a 









‘3 






o 



‘3 


‘C 


o 




‘3 


o 



!Z3 

’S3 



£ 

o 

a 



[a 

13 

3 

a 

o 

a 


a 

5- 

3 


Dh 

a 

5-i 


£ 

o 

a 

’a 

‘3 

o 


a 

a 

.a 

13 

5-i 

3 

Dh 

3 

5-< 


rt 

Id 

a 


[a 


D- 

6 

5-h 






o 



o 

a 

o 


a 


o 


o 


a 

a 

O 


'a 

G 


o 


O 



Q 



P 



00 

Z 

P 


z 


Z 


P 

00 


P 

z 

Z 


a 


00 


Z 






© 



a 

a 

a 


© 




© 

© 


a 

a 



CD 


a 





a 

£ 


a 



a 

a 

a 


a 


a 


a 

a 


a 

a 



a 


a 





h-> 



a 



a 

a 

a 


a 



a 

a 


a 

a 



a 


a 





5/3 



•a 



~o 

TD 

•a 


•a 


G 


T3 

•a 


■a 

■a 



■a 


as 






o 


a 



a 

a 

a 


a 


o 


a 

a 


a 

a 



a 


a 





a 

La 

S3 

& 



& 

& 

& 


& 


P 


& 

& 


& 

& 

H_> 


& 


£p 





a 

O 

.© 

a 



iD 

ID 

a 


a 


© 


ID 

a 


a 

a 

a 


a 


a 








-o 



as 

-o 

•a 


•a 




X 

•a 


■a 

-a 



-a 


■a 





a 

5/3 

a 

a 



a 

a 

a 


a 


O 


a 

a 


a 

a 

3 


a 


a 


a 



5a 

a 

a 

P 



P 

P 

P 


P 


Z 


P 

P 


P 

P 

P 


P 


P 

















a 




























a 


























in 


o 







in 






o 



CD 



CN 



CN 

CN 

CN 


CN 


Dh 


CN 

CN 


CN 

CN 

CN 


CN 


CN 


a- 






oo 



OO 

OO 

00 


CN 


a 


00 

OO 


OO 

OO 

CN 


OO 


OO 


P 












CN 


5-i 







CN 






CO 















o 




























Z 




























a 
















Lh 



CD 



u 

^© 



<D 


a 


ID 




a 



^D 


^D 





V 



a 



"a 

13 

_© 


a 


o 

Dh 


a 

JD 


u 

13 



13 


"a 


Jh 



a 



£ 

CD 



a 

<D 

a 

<D 

a 

2 


a 

ID 


3 


£ 

a 

a 

2 


13 

s 

a 

<D 

a 

2 


a 

iD 


a 

ID 


13 

2 



O 



P 



0- 

p 



P 


o 


Oh 




P 



P 


P 

















Z 












































*- 



*o 




vo 

_ _ ( 


vo 


i-H 


X 

»—i 


X 

-H 

X 


X 


-H 


X 



© 



r- 



oo 

oo 

ON. 


o\ 


o 


O 




CN 

CN 


cn 


a- 


a- 



p 































Group 5 (email without warnings, cut and paste attack): De¬ 
mographics 


226 


.3 

© ss 
15 l 
£ 


as 


a 

x 

W 


.2 1 

c ° 

a $ 
•2 © 
"O 


CQ 


cx 

& 

-C 


CQ < 


=3 

< 


O- 

O 


Oh 

& 

-C 


CQ 0-. 


CQ < 


03 


cx 

% 

-C 


CQ < 


CQ 0-i 


I I 


13 


a- 

P- 

< 


O 


©X) 

< 




Group 5 (email without warnings, cut and paste attack): De¬ 
mographics 


227 


0 £ 

a 

6 

a 
























© 

5/3 
























■a 

a 

a 












5/3 


5^ 









5^ 

© 

s- 

© 

a 




.a 




5/3 

X 2 



X 


.5 

■s 








X 







a 







a 


a 

a 








a 


[a 

"a 

© 

c3 

© 



o 

a 


c3 

© 


a 

o 

a 


y 

© 


o 

a 


o 

a 

o 

a 


H 

a 

© 


c3 

© 


c3 

© 


o 

g 

© 

a 

5/3 

>> 



Pt 





>> 


(N 


(N 

<N 





© 

>> 


(N 

& 

a 

a 

CN 





<N 


so 

<N 

| 







(N 

| 


CN 

> 

CN 



X 

a 

© 

A 



1 


A 




1 


1 

1 



A 

© 

A 


1 

w 

cC 




SO 



V 



SO 


SO 

so 




Z 


VO 

5/3 

r- 



bp 



bp 


bp 


bp 

bp 


bp 



bp 


bp 




bp 



# © 



_£ 

a 


a 


a 


a 

a 


a 



a 


a 




a 



*-3 



‘a 

.2 


" 3a 


'Sa 


"Sa 

'Ba 


'Sa 



'Ba 


'Ba 




'Ba 



© 



© 

© 


Pa 


Pa 


o- 

Oh 


Pa 



Pa 


Pa 




Pa 


bp 

cs 



> 

a 


O 


O 


o 

O 


O 



O 


O 


a 

a 

O 



5/3 

a 

C3 



£ 

< 


-a 

00 


X 

00 

13 

da 

x 

00 

X 

00 


-a 

00 

13 

Cd 

13 

da 

00 

13 

cu 

-a 

00 

13 

da 

_o 

© 

_o 

© 

J3 

00 

13 

cu 

'Ba 

Q, 















>> 





>, 

a 

a 


>> 

o 

a-- 

V 

a 








a 

da 





a 

Cd 

a' 

CU 


a 

CU 


a 

da 

< 

< 


a 

Cd 

X 

00 

"a 



, 

bp 


„ 



& 



„ 

5/1 

„ 

5« 


„ 

5/T 


5/f 

bp 

bp 


5/T 



a 


bp 

a 


bp 


bp 

c 

bi 

3 

bp 

£3 

bp 

c 

bp 

bp 

££ 

bp 

C 

_a 

a 

bp 

£3 

bp 

*o 

£5 

© 


a 

'B- 

a 

a 

a 

a 

O 

a 

a 

a 

_o 

a 

_o 

a 

a 

_o 

a 

_o 

‘Ba 

'B- 

a 

_o 

a 

a 

© 


£ 

o- 

da 

£ 

da 

£ 


£ 

da 

£ 


£ 


£ 

£ 


£ 


Pa 

Pa 

£ 


£ 

£3 

a 



o 

>, 

a 


a 

© 


>» 


© 

a 

© 

a 


© 

a 

© 

o 

o 


© 

a 


© 



X 

a 

a 

a 

a 

a 


a 


3 

a 

3 

a 


a 

a 

3 




a 

a 

£ 

-a 


CQ 

00 

Cd 

CQ 

da 

CQ 

< 

CQ 

da 

QQ 

< 

CQ 

< 

CQ 

CQ 

< 

CQ 

< 

00 

00 

OQ 

< 

CQ 



























© 

























© 



























o 



o 







o 


o 



O 


O 

o 




■§ 

"25 

5/3 

La 

CN 



CN 


+ 


o 

o 


<N 


<N 

+ 


<N 


<N 

<N 

+ 


o 

& 

5/3 

JZ 

O 



O 


O 





O 


O 

o 


O 


O 

O 

o 



a 


’ 1 



1—1 


<N 


in 

in 





<N 






(N 


in 

















X 






X 



















PJ 






W 



>» 

l. 
























u 

a 

£ 

© 


X 



X 


X 


X 



X 



© 


X 


X 

X 

© 


X 

1 


£ 



£ 


cS 


£ 

1 


£ 


‘C 

a 


£ 


£ 

£ 

g 

<3 

52 


© 


© 



© 


© 


© 


© 


,a 

© 

L-. 

© 


© 

© 

© 

La 

© 

£ 

La 


La 



_La 


La 


La 

a 


•— 


a 


o 

_L- 


_L- 

La 


O 

La 

On 

X 


£ 



£ 


£ 


£ 

00 


£ 


00 

a 

■p- 

£ 


£ 

£ 

a 

Da 

£ 

C/3 



£ 

o 



5/1 

£ 

o 


00 

O 


% 

o 

00 

O 


5/1 

£ 

o 


00 

O 

% 

c 


00 

O 


00 

O 

% 

o 

c 


5/1 

£ 

o 



T3 



■a 


© 


T3 

© 


T3 


© 

T3 


© 


© 

T3 

T3 


T3 

o 



a 



a 


a 


a 

a 


a 


a 

a 


a 


a 

a 

a 


a 




s 





2 


s 

2 




2 

s 


2 


2 

5 

5 

























© 

























a 









© 




© 



© 





© 



© 

a 

X) 






© 



© 




© 





© 

© 


© 



© 



c3 




o 



a 




a 



3 


© 

© 


a 



a 

o 


_a 




a 



_© 




_© 



o 


a 

a 


_© 



_© 







© 

'© 



‘© 


© 


‘© 

_© 


© 

La 


_© 

‘© 

© 

’© 


’© 


s 

‘© 

© 

S 

o 

a 

o 


[Ha 

‘© 

‘a 

a-> 



•Q 



13 

La 

a 


'a 

a 

s 


13 

La 

a 

a 

a 

s 


13 

•— 

a 


13 

*o 

[a 

‘© 


13 

•— 

3 


‘a 

a 

a 

a 

Lh 

a 


'O 

La 

© 

0) 



o 



a 


3 


a 

3 


a 

a 


o 

o 


a 


3 

t3 

© 



Q 



00 



Z 


ffi 


Z 

X 



00 

00 


Z 


s 

Z 

W 


_a 




© 



© 


© 


© 

© 


© 


© 

© 


© 


© 

© 

© 


© 

a 

£ 


a 



a 


a 


a 

a 


a 


a 

a 


a 


a 

a 

a 


a 

a-> 



a 



a 


a 


a 

a 


a 


a 

a 


a 


a 

a 

a 


a 

5/3 



•a 



T3 


~o 


T3 

T3 


T3 


■o 

T3 


~o 


T3 

•a 

T3 


T3 


© 


a 



a 


a 


a 

a 


a 


a 

a 


a 


a 

a 

a 


a 

a 

La 

a 

& 



& 


& 


& 

& 


& 


& 

& 


& 


& 

& 

& 


& 


o 

.© 

© 



© 


© 


© 

© 


© 


© 

© 


© 


© 

© 

© 


© 




T3 



T3 


T3 


-a 

T3 


T3 


T3 

-a 


T3 


-a 

T3 

T3 


-o 

a 

5/3 

a 

a 



a 


a 


a 

a 


a 


a 

a 


a 


e 

a 

a 


a 

55 

a 

a 

P 



P 


D 


P 

P 


D 


D 

P 


P 


P 

P 

P 


P 




o 












in 










Age 



LT) 



<N 


CN 


<N 

<N 


CN 


CN 

CN 


(N 


<N 

Cd 

<N 


(N 



P 



oo 


OO 


CO 

OO 


OO 


(N 

OO 


OO 


OO 

00 

OO 


OO 















CN 










La 



© 





J© 


© 

© 


£ 


£ 

© 


Si 


£ 


© 


s 

V 



a 



^© 


13 


13 

a 


a 


13 

a 


13 


13 

_© 

a 


13 

a 



s 



13 


a 


a 

a 


s 


s 

a 


a 


a 

13 

s 


s 

0) 



© 





© 


© 

© 


© 


© 

© 


© 


© 


© 


© 

O 



da 





tt. 


da 

da 


da 


da 

da 


da 


da 


da 


da 


























*— 



SO 



— 


SO 


.—i 

SO 


-H 


SO 



SO 


00 

— 

SO 



© 






<N 


<N 


m 

m 


at" 


a- 

in 


in 


SO 

t"* 

r- 


OO 

5/3 



<N 



<N 


<N 


<N 

CN 


<N 


CN 

CN 


<N 


CN 

(N 

<N 


CN 

P 




























Group 5 (email without warnings, cut and paste attack): Web 
security attitudes 


228 


-d 

3 


© a 


3 

o oid 
33 s 

§ 'I 

o> o 

r? 

a ^3 


a 

a — 


£ 

js !> 

OX) 

3 - 




a 
3 J* 

3 j CU 


CO 

ec 


S- 

£ 

.3 


a a 


i * 

3 3 

V © 

u +* 

cu .S 


£ -5 


O CL, £ 

© £T o 

^ g 

<D 

s-< c« c« 

O p « 

S i 1 

D .3 a) 

3 £ £ 


■a 


o 

op 


c 

'rB 


-3 

H 


s I 


£ 

o 

3 


m ^ a .% 


& -s 


+3 .3 


3 

£ 

o 

T3 


£ 

u 

H 

<c 


3 

‘5b 


OP 3 o op ^4 


£• & 


e .s 


.0 .0 0 


■- £ ■- 


S -2 


a 

6 

o 




Group 5 (email without warnings, cut and paste attack): Web 
security attitudes 


229 



-C 

£ 


£ 

>> 


Oh 

£ 


£ 

<D 




Group 5 (email without warnings, cut and paste attack): Web 
security attitudes 


230 


-d 

s 


© a 


& 

o oid 
33 S 

§ 

o 

r? ^ 

a, m 




a 

a — 


2 !> 
OX) 

s- 




=3 


a 
a g* 

3 J cu 


CO 

pa 


3 

O- 

£ 


a a 


i * 

CO CO 

V © 
s- t3 

eu .S 


^ o 

3 £ 


is 

2 


o 


£ £ 


.®2 >p 


c 

Co 

is 


* & 


2 .5 


£ a 


> 

•c 

D- 


£ 

2 

cs 

c 


S 5 










Group 5 (email without warnings, cut and paste attack): 
Study experiences 


233 



curity breach since I was only making movie 
predictions. 




Group 5 (email without warnings, cut and paste attack): 
Study experiences 


234 







Group 5 (email without warnings, cut and paste attack): 
Study experiences 


236 


.5 6 ® 


CQ 

0 C 


CQ 

u 

D 


> 

‘5b 




.S w 
a> "d 


DJ5 S3 


JS OX) 

« 5 

p- CO 

i 1 

11 

c/3 a 


— co 
4 ) Jj 
CO 







Group 5 (email without warnings, cut and paste attack): 
Study experiences 


238 





Group 5 (email without warnings, cut and paste attack): Reg¬ 
istration attitudes 


239 



















Group 5 (email without warnings, cut and paste attack): Reg¬ 
istration attitudes 


244 


Convenience of: 

Passwords + email 

for registration 

I could get used to it 

I could get used to it 

I could get used to it 

I don’t know 

I could get used to it 

1 hardly noticed it 

1 hardly noticed it 

Passwords + no reg¬ 
istration 

I hardly noticed it 

I could get used to it 

I hardly noticed it 

I don’t know 

I hardly noticed it 

I hardly noticed it 

I hardly noticed it 

Security of: 

Passwords + email 

for registration 

Very secure 

Fairly secure 

Very secure 

I don’t know 

Very secure 

I don’t know 

Fairly secure 

Passwords + no reg¬ 
istration 

Somewhat secure 

Somewhat secure 

Fairly secure 

I don’t know 

Fairly secure 

I don’t know 

Somewhat secure 


Benefits of email registration 

It just adds another step in verifying 

that you are who you say you are and 

thus helps prevent other people from 

logging into your account. 

makes sure that the person registering 

is a real person? 

It provides only the user with the pass¬ 
word of the email to register their 

own computer so information cannot 

be stolen. 

It forces the person with the e-mail 

address to confirm their account, you 

therefore cannot pose as other people. 

(Or you could only do so with their so- 

called consent.) 

I believe that when a person is signed 

up with an email it not only has to be 

a real email address but the person has 

to go to their email and confirm that 

it is their’s. I assume this limits some 

identity theft. 

I don’t really know 

No response 

Used email registra¬ 
tion before? If yes, 

where? 

yes: experimetrix 

(Berkeley psychology 

labs) and others that I 

don’t remember 

no 

yes: pay pal, ebay 

yes: Numerous 

yes: facebook 

no 

yes: multiple 
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Group 5 (email without warnings, cut and paste attack): 
General questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

76 

0-5 minutes 

No 

yes 

The study saids I have to make predic¬ 
tions for 7 days but I’ve only made pre¬ 
dictions for 4 days and then I’m already 

completing the exit survey. That means 

I don’t have the chance to make more 

money by predicting for more days. 

81 

5-10 minutes 

No, I enjoyed it. 

yes 

Interesting study. 

86 

0-5 minutes 

no 

yes 

no 

91 

0-5 minutes 

No. 

yes 

No. 

96 

0-5 minutes 

Nothing. 

no 

No. 

101 

No response 

no 

No response 

No response 

106 

0-5 minutes 

No. 

yes 

No response 

111 

0-5 minutes 

The study said I was going to make 7 

predictions and then gave me the exit 

survey after only 5. That was annoy¬ 
ing because I am now worried I won’t 

get credited for completing the whole 

study. 

yes 

It was fairly enjoyable and easy to ac¬ 
cess thanks to its distribution online. 

116 

5-10 minutes 

no 

yes 

No response 

121 

5-10 minutes 

No response 

yes 

No response 

126 

0-5 minutes 

no 

yes 

nope 

136 

5-10 minutes 

No 

no 

No response 

141 

0-5 minutes 

No. 

no 

No. 

146 

0-5 minutes 

no 

yes 

No response 

151 

0-5 minutes 

No response 

yes 

Great trick. 

156 

0-5 minutes 

Not really, maybe the design of the 

actual webpage. It looked a little 

amateur-ish. 

yes 

I liked predicting movie box office 

grosses. Movies interest me, so this 

study was interesting to me, even 

though it seems to be less about predic¬ 
tions now, and more about Internet se¬ 
curity. 

161 

0-5 minutes 

I wanted to base my future predictions 

on previous data, but that wasn’t al¬ 
ways available. 

yes 

It ended a few days earlier than I ex¬ 
pected it to. 

166 

0-5 minutes 

No 

yes 

It was fun trying to guess everyday. 

171 

0-5 minutes 

No. 

yes 

It was very deceptive. 

176 

0-5 minutes 

The multiple registrations (or was that 

me being tricked...) was kind of an¬ 
noying. 

yes 

Thanks for letting me participate! Hope 

this helps. 

181 

5-10 minutes 

No response 

yes 

No response 
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Group 5 (email without warnings, cut and paste attack): 
General questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

186 

5-10 minutes 

No response 

yes 

No response 

191 

0-5 minutes 

no 

yes 

nope 

196 

5-10 minutes 

No 

yes 

No response 

201 

5-10 minutes 

No it was very easy. 

yes 

No response 

206 

0-5 minutes 

No 

yes 

No response 

211 

0-5 minutes 

Not really. 

yes 

I didn’t make the full 7 predications, but 

somehow triggered the end of the study 

anyways. Not sure if this is intended 

behavior. 

216 

0-5 minutes 

No. I thought predicting high-grossing 

movies was fun. 

yes 

no. 

221 

0-5 minutes 

No response 

yes 

No response 

226 

0-5 minutes 

Waiting for e-mail confirmation was 

annoying. It didn’t say how long to 

wait for the confirmation, so I was go¬ 
ing to wait a few hours until I decided 

to e-mail you about the problem, but 

that was cleared up within the hour 

since you sent an e-mail explaining the 

website was having server problems in 

sending out registration e-mails. 

yes 

So I take it you’re not drafting up a 

set of statistics of how well/badly we’re 

churning out movie predictions. :( 

231 

0-5 minutes 

the registered computer thing. 

no 

no 

236 

0-5 minutes 

Email registration 

yes 

No response 

241 

0-5 minutes 

no 

yes 

no 

246 

5-10 minutes 

Nope, I really liked it. 

yes 

No response 

251 

0-5 minutes 

no 

no 

No response 

256 

0-5 minutes 

no, it was very easy 

yes 

explain that there is a rated order of the 

movies and it affects how much money 

you get 

268 

5-10 minutes 

No. 

yes 

It’s easy to look up box office predic¬ 
tions, ratings, and potential forecasts on 

the internet. Nowhere in the study does 

it ask a person to refrain. Addition¬ 
ally, newspapers are a good resource 

and one’s own family if they work in the 

movie industry. 

271 

5-10 minutes 

no 

yes 

It was difficult to remember that I was 

signed up for the study. Perhaps con¬ 
sider sending daily reminders to do the 

study would be beneficial 
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Group 5 (email without warnings, cut and paste attack): 
General questions/feedback 


User # 

Average length 

of each visit 

Anything annoying or difficult? 

Engaging or 

interesting? 

General comments 

276 

0-5 minutes 

no 

yes 

I felt pretty dumb after I’d been duped. I 

think if there was an obvious way to by¬ 
pass the registration at the end I might 

not have done it but it seemed like I 

didn’t have a choice. 

281 

0-5 minutes 

no 

yes 

No response 



















